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Foreword 


The 2016 cycle of the International Civic and Citizenship Study (ICCS) is the fourth 
International Association for the Evaluation of Educational Achievement (IEA) study 
examining the ways in which young people are prepared to undertake their roles as 
citizens. The IEA first studied this realm in 1971 as part of the Six Subject Survey in nine 
countries, which led to the 1999 IEA Civic Education Study (CIVED) conducted in 28 
countries. Both studies were explicit recognitions that foundational skills are important, 
yet that these alone are not sufficient for prospering in today’s global society, and in a 
world that requires an open and more culture-oriented approach, a moral orientation 
emphasizing human rights, and a focus on social justice and political change. 


Recognizing the urgent need for continued international research and evidence 
on topics related to civic and citizenship education, ICCS 2009 was established as a 
baseline study for future assessments. With 38 countries participating from around the 
globe, data gathered from more than 140,000 Grade 8 students and 62,000 teachers 
in over 5300 schools provided evidence that revealed considerable variation among 
and within countries about the provision of civic and citizenship education, as well 
as civic knowledge of students, but also indicated that large majorities of students in 
all countries strongly endorse democratic values and institutions. The comprehensive 
core assessment was complemented by three regional modules for Asia, Europe, and 
Latin America, designed to flexibly recognize local interest and to investigate cultural 
aspects of civic and citizenship education, particularly in light of the growing impact 
of the processes of globalization and changing contexts of democracy and civic 
participation. Finally, a civic and citizenship education encyclopedia, a technical report, 
and an international database accompanied by a comprehensive user guide allowed the 
broader research community to use the ICCS data for in-depth analyses. 


The 2016 cycle of ICCS will build on the data gathered in ICCS 2009. This publication 
presents the ICCS 2016 assessment framework, which provides a conceptual 
underpinning to the measurement of antecedents, processes, and outcomes of civic 
and citizenship education undertaken in the second cycle of this study. A central aim is 
to monitor changes in students’ civic knowledge and engagement over time by linking 
the second survey cycle directly to ICCS 2009, allowing the countries that participate 
in both cycles to monitor trends in civic knowledge and engagement over seven years. 
Moreover, there is the equally important need to reflect and address new and emerging 
challenges to improve countries’ understanding of issues such as students’ role with 
respect to environmental sustainability, social interactions at school, and the use of new 
social media for civic engagement. As in the 2009 ICCS survey, the current study allows 
countries to explore specific topics of common interest, through the establishment of 
regional modules in Europe and Latin America. Such reliable, comparative evidence 
and data will enable them to evaluate the strengths of educational policies, both 
internationally, and within regional contexts, and to measure progress in achieving set 
goals. 


More recently, Global Citizenship Education (GCED) and Education for Sustainable 
Development (ESD) were identified as critical components of the post-2015 development 
agenda, expressed as Target 4.7 of the United Nations’ Sustainable Development Goals 
(SDGs) (United Nations, 2015). 
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IEA analysts contributed to the UNESCO-led development of indicators for monitoring 
progress at the global, regional and national levels, and to discussions in the global 
citizenship education working group of the Learning Metrics Task Force (LMTF). 
Building on our experience and expertise in studying civic knowledge and citizenship 
education, the IEA carried out a study and suggested a list of potential indicators for 
GCED and ESD for post-2015 monitoring. In 2016, UNESCO and the IEA established 
an official agreement to collaborate in the area of measuring GCED and ESD. We are 
proud to be involved in this vital global mission and will continue to explore these and 
related data needs with UNESCO; ICCS is one of the major existing sources of data. 


Drawing on an established international network of research organizations, scholars 
and technical experts, two partner institutions, in cooperation with the IEA, and the 
national research centers (NRCs), are responsible for the study’s organization and 
implementation. These are the international study center at the Australian Council for 
Educational Research (ACER), and the associate research center at the Laboratorio di 
Pedagogia Sperimentale (LPS) at the Roma Tre University in Italy. I thank the research 
teams for their intellectual leadership, dedication and support: namely, John Ainley, 
Julian Fraillon, Tim Friedman, and Wolfram Schulz from ACER; and Gabriella Agrusti 
and Bruno Losito from LPS. My special thanks go to the members of the Project 
Advisory Committee (PAC), for their thoughtful reviews of the framework and scholarly 
guidance during the study’s development: namely Erik Amnå (Örebro University, 
Sweden), Cristián Cox (University Diego Portales, Chile), Barbara Malak-Minkiewicz 
(IEA Honorary Member, The Netherlands), Judith Torney-Purta (University of 
Maryland, United States), and Wiel Veugelers (University of Humanistic Studies, The 
Netherlands). 


I further thank the key research, management, and support staff at the study’s 
coordinating center at the IEA DPC, namely Falk Brese, Ralph Carstens, Marta Kostek, 
Hannah Köhler, and Sabine Weber, as well as at the IEA Secretariat, namely Paulina 
Koršňáková and Gabriela Noveanu, for their leadership and tireless commitment to 
the success of the project, and Roel Burgers and Isabelle Gemin. The IEA Publications 
and Editorial Committee (PEC) suggested improvements to earlier versions of the 
framework and my thanks go to Seamus Hegarty on behalf of the group, and Gillian 
Wilson, who edited the document. 


Critical funding has been provided by the European Commission Directorate-General 
for Education and Culture in the form of support grants to the European countries 
participating in the project and, of course, by the 24 participating countries and 
education systems. 


As always, this IEA study is dependent on the critical input, perseverance, and 
enthusiasm of the NRCs and their staff. From their collaboration on the development 
of the framework, to the meticulous management and execution of the study at the 
national level, their sustained contributions are what ensure a truly successful venture. 
They are both the foundation and our guides in all the IEA’s endeavors. 


Dirk Hastedt 
EXECUTIVE DIRECTOR IEA 
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Overview 


1.1 Purpose of the study 


The purpose of the International Civic and Citizenship Education Study (ICCS) is to 
investigate the ways in which young people are prepared to undertake their roles as 
citizens in a range of countries in the second decade of the 21th century. ICCS 2016 is a 
continuation of this study, which was initiated in 2009. The first ICCS survey reported 
on student achievement using a test of conceptual knowledge and understandings of 
aspects of civics and citizenship. It also collected and analyzed data about student value 
beliefs, attitudes, behaviors and behavioral intentions relating to aspects of civics and 
citizenship. 


In recognition of the need for continuing research on civic and citizenship education 
and the widespread interest in the establishment of regular international assessments of 
civic and citizenship education, the IEA (International Association for the Evaluation 
of Educational Achievement) decided to undertake a second study cycle of ICCS with a 
data collection in 2016. The second ICCS survey is intended to respond to enduring and 
emerging challenges of educating young people in a world where contexts of democracy 
and civic participation continue to change. New developments of this kind include the 
increase in the use of social media by young people as a tool for civic engagement, 
the growing concerns about global threats and sustainable development, as well as 
spreading recognition about the role of schools in fostering peaceful ways of interaction 
between young people. 


Furthermore, civic competencies can also be viewed as an essential part of a broader 
skill set required in workplaces, and thus these competencies are not only of interest to 
political and community leaders, but are also valued by a growing number of employers 
(Gould, 2011). There is an increased recognition by leaders of the business community 
that technical skills are important, but that these skills are not sufficient for prospering 
in today’s global economy. Consequently, it is to be expected that employers in the 21st 
century will be seeking to hire and promote individuals with ample knowledge about 
significant changes in society, intercultural literacy, ethical judgment, humanitarian 
values, social responsibility, and civic engagement (OECD [Organisation for Economic 
Co-operation and Development], 2015). 


ICCS 2016 will allow both the measurement of changes over seven years (from 2009 
to 2016) and the assessment of additional aspects of civic and citizenship education, 
including those related to recent developments in a number of countries. The ICCS 
instruments include a large range of test and questionnaire material from the previous 
study, which permits the comparison of changes in civic knowledge, attitudes and 
engagement over time. In addition, new item material was developed to measure aspects 
that were not included in ICCS 2009. It is expected that future ICCS cycles will take 
a similar approach, where instruments include both old and new material to permit 
comparisons over time at national and international levels, as well as the measurement 
of additional cognitive or affective-behavioral aspects. 


© International Association for the Evaluation of Educational Achievement (IEA) 2016 
W. Schulz et al., LEA International Civic and Citizenship Education Study 2016 
Assessment Framework, DOI 10.1007/978-3-319-39357-5_1 
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1.2 Study background 


Prior to ICCS 2016, the IEA conducted three international comparative studies about 
civic and citizenship education, with a first survey implemented in 1971, a second one 
in 1999/2000 and third one in 2008/2009 (Schulz, Fraillon, & Ainley, 2011; Torney- 
Purta, & Schwille, 2011). 


The first IEA study concerning civic and citizenship education was undertaken in 
1971 as part of the Six Subject Study (for a summary, see Walker, 1976). Ten countries 
participated in this data collection and the report was published in 1975 (Torney, 
Oppenheim, & Farnen, 1975). The study included assessments of civic knowledge 
among 10- and 14-year old students and collected questionnaire-based data from 
students, teachers and school principals. 


The second study, the IEA Civic Education Study (CIVED), was carried out in 1999. It 
was designed to strengthen the empirical foundations of civic education by providing 
information about the civic knowledge, attitudes, and actions of 14-year-olds. The study 
had a twin focus on school-based learning and on opportunities for civic participation 
outside the school. It focused on three civic-related domains: democracy / citizenship; 
national identity / international relations; and social cohesion / diversity. The focus 
on these domains was chosen as being particularly useful to policymakers involved in 
designing or redesigning curricula and preparing teachers. 


CIVED was highly successful in achieving its aims and objectives, and established the 
evaluation of student outcomes in this learning area as an integral part of international 
comparative educational research. Phase 1 produced a detailed series of national case 
studies from the 24 participating countries (Torney-Purta, Schwille, & Amadeo, 1999). 
Phase 2 produced two data-rich international reports, the first on the results from the 
mandatory standard population of 14-year-olds in 28 participating countries (Torney- 
Purta, Lehmann, Oswald, & Schulz, 2001) and the second from the 16 countries 
that surveyed an older population of 16- to 18-year-olds (Amadeo, Torney-Purta, 
Lehmann, Husfeldt, & Nikolova, 2002). CIVED findings have had a considerable 
influence on policy and practice in civic and citizenship education across the world, 
in both participating and non-participating countries, and have also influenced 
further (national and international) research in this area (Kerr, Ireland, Lopes, Craig, 
with Cleaver, 2004; Mellor, & Prior, 2004; Menezes, Ferreira, Carneiro, & Cruz, 2004; 
Reimers, 2007; Torney-Purta, 2009). 


ICCS 2009 was built on previous IEA studies of civic education, particularly the CIVED 
study conducted in 1999 (Amadeo et al., 2002; Torney-Purta et al., 2001), and was also 
established as a baseline study for future assessments in this learning area. Like its 
predecessor, it included a student test of civic knowledge and understanding, as well as 
questionnaires for students, teachers and school principals. However, there were some 
notable changes regarding the design and scope between CIVED and ICCS 2009: 


+ The assessment framework was broadened to: (i) have a stronger focus on the 
motivations for, and mechanisms of, participation associated with citizenship 
education, (ii) include a wider range of content, and (iii) place a greater emphasis on 
reasoning and analyzing in addition to knowing. 

+ The civic knowledge test was administered using a balanced rotated design of seven 
booklets (including one with CIVED link items) so as to assess a wider range of 
content and provide for a more extensive coverage of thinking processes. 
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+ While CIVED collected data from two or more teachers of civic-related subjects 
in the selected class, the ICCS 2009 teacher survey was directed toward all teachers 
teaching the target grade in selected schools and thus collected data from a larger 
and more representative sample at each selected school. This change recognized the 
influence of the school environment on civic-related learning outcomes. 


* To supplement existing published information at country level, additional data on 
national contexts were collected through a questionnaire completed by national 
centers drawing on expertise in each of the participating countries. 


* For the first time in IEA history, ICCS 2009 established regional modules for three 
geographic regions (Asia, Europe and Latin America) that included the development 
of additional student instruments addressing specific aspects relevant to each region. 


+ An encyclopedia was produced that consisted of detailed descriptions of national 
contexts, policies and curricula relating to civic and citizenship education for 
countries that had participated in the study (Ainley, Schulz, & Friedman, 2013). 


The results of this study were reported in a series of IEA publications (Fraillon, Schulz, 
& Ainley, 2012; Kerr, Sturman, Schulz, & Burge, 2010; Schulz, Ainley, Fraillon, Kerr, & 
Losito, 2010a, 2010b; Schulz, Ainley, Friedman, & Lietz, 2011). Analyses of data from 
ICCS 2009 have also led to numerous reports and publications within countries, as well 
as reports and papers based on secondary research. 


1.3 Recent developments and persisting challenges 


Since the inception and implementation of ICCS 2009, a number of new global 
developments that have implications for civic and citizenship education across many 
countries have occurred: 


* The global financial crisis of 2007—2008 and the global recession that followed 
have had a strong impact on many societies, and underlined the importance of the 
economy for social cohesion and political stability (Chossudovsky, & Marshall, 2010; 
Grant, & Wilson, 2012; Shahin, Woodward, & Terzis, 2012). In particular, in those 
countries that were hit hardest by recession, budget deficits and subsequent austerity 
measures have been characterized by ongoing dissent about economic policies, high 
unemployment rates (especially among young people), and the emergent success of 
protest parties and movements at elections. Since 2011, there has been an economic 
recovery in a number of countries, while others have experienced a worsening of 
economic and social conditions. These developments in the economic sphere have 
consequences for the ways in which citizenship education is envisioned (Kennedy, 
2012). 


+ In many societies, the potential impact of human activity on the environment 
(in particular on the global climate) as well as the question about the long-term 
sustainability of development have increasingly become key issues in debates about 
their future political, social and economic development (Dringer, 2013; IPCC 
[Intergovernmental Panel on Climate Change], 2014). Under the auspices of the 
United Nations there have been successive Climate Change Conferences since 1995 in 
Berlin with recent conferences in 2009 (Copenhagen), 2014 (Lima) and 2015 (Paris). 
Regard for the environment and its long-term protection are increasingly regarded 
as integral parts of responsible citizenship with implications for the development of 
civic and citizenship curricula (Lotz-Sisitka, Fien, & Ketlhoilwe, 2013). 
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+ In many countries there are also increasing concerns about how schools can ensure 
peaceful coexistence within school communities. In particular abuse and bullying of 
students (by other students and often aimed at various types of social minorities) 
have become salient issues in discussions about schools and learning environments. 
The recent movement of large numbers of refugees from the Middle-Eastern region 
to other (mostly European) countries will most likely increase the need for integrating 
people from different backgrounds into society. This will also result in challenges 
to schools in relation to their functioning as socially heterogeneous communities. 
In many countries, civic and citizenship education includes goals related to the 
promotion of student engagement with a school community (Fredericks, Blumenfeld, 
& Paris, 2004; Willms, Friesen, & Milton, 2009), fostering a peaceful coexistence and 
providing students with mechanisms for conflict resolution (Johnson, & Johnson, 
1996; Mickelson, & Nkomo, 2012). 


* Continuing development of information and communications technologies (ICT) 
has led to an increase in the use of ICT and new social media for civic participation. 
New social media played an important role in initiating and maintaining support as 
part of the revolutionary protests in the Middle East, in promoting action on climate 
change or in organizing protests against austerity measures in the aftermath of the 
global financial crisis (see for example Kahne, Middaugh, & Allen, 2014; Milner, 
2010; Segerberg, & Bennett, 2011). 


There are also persisting challenges to the study of civic and citizenship education, 
which have an ongoing impact on civic and citizenship education across participating 
countries: 


* The concept of democracy is intrinsically linked to the notion of citizenship in 
the sense that democratic processes are generally conceived as dependent on an 
informed and active citizenry. Most countries around the world consider themselves 
as democracies, but in many of them there are concerns about the real state of 
democratic process due to the exclusion of larger parts of the population and the 
erosion of liberties (Diamond, 2015; Kagan, 2015). Furthermore, in countries 
with long-standing democratic traditions, there has been evidence of a general 
downturn in citizenship participation, in particular among younger people (see for 
example, Dalton, 2002; Putnam, 2000). While education is widely recognized as an 
important tool for fostering democratic citizenship (Naval, Print, & Veldhuis, 2002), 
there are different approaches to citizenship education, which may depend on the 
underlying basic conceptions of democracy. For example, Westheimer and Kahne 
(2004) distinguished promoting personal responsibility, active participation, and 
justice orientation as different approaches in this learning area, and Veugelers (2007) 
identified adapted, individualized and critical-democratic orientations in civic and 
citizenship education. 


* Increasing globalization continues to influence debate about citizenship education 
because it challenges traditional notions of citizenship (which have been linked to 
the nation state), and it has led to supra-national concepts of (global) citizenship 
and new forms of experiences with citizenship across borders as a result of migration 
(Brodie, 2004; O'Sullivan, & Pashby, 2008; Reid, Gill, & Sears, 2010; Schattle, 2012; 
Torres, 2002). Based on research among teachers, Veugelers (2011) distinguished 
three different interpretations of global citizenship: an open and more culture- 
oriented approach, a moral orientation emphasizing cosmopolitism and human 
rights, and an approach focusing on social justice and political change. 
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ICCS 2009 collected a rich data set to support comparative analyses of civic and 
citizenship education and provided many interesting results. However, given that the 
age group was 13- to 14-year-olds, there were some limitations on the extent to which 
their knowledge, perceptions and behaviors could be assessed. The following issues 
were taken into account in the development and refinement of the instruments for 
ICCS 2016: 


* Results from CIVED and ICCS 2009 showed that students at this age tended to report 
little inclination to engage in conventional forms of political or civic participation. In 
particular, expectations of active engagement in politics through parties, trade unions 
or local elections were generally low and tended to be associated with lower levels 
of civic knowledge. Therefore, ICCS 2016 places more emphasis on aspects closer 
to young people’s interests and possibilities of engagement when it is measuring 
students’ attitudes, behaviors or behavioral intentions. 


* Questions about democratic beliefs tended to be endorsed by overwhelming 
majorities of students and therefore provided little information about differences 
in attitudes among young people of this age. Moreover, in the Latin American 
region, the responses contradicted more general findings about positive responses 
to statements, such as those endorsing dictatorships provided they bring benefits 
to society (see Schulz, Ainley, Friedman, & Lietz, 2011). In ICCS 2016, when asking 
student about their views on democracy, preference was given to beliefs that were 
not necessarily prevalent in their society, such as government influence on courts or 
nepotism. 


1.4 Broadening the scope of ICCS 


The development of a framework for ICCS 2016 needed to take account of recent 
developments and ongoing challenges. To achieve this, the international project team, 
in close cooperation with experts and country representatives, identified areas related 
to civics and citizenship education, which had either gained more attention in recent 
years or were regarded as relevant, but which were not addressed in great detail in the 
previous ICCS survey. Each of these areas includes some aspects that were included in 
ICCS 2009 to a certain degree, but are addressed in a more comprehensive and broader 
form in the current study. Through this approach, the wider conceptual framework for 
ICCS is also open to including additional aspects in future cycles. 


The following three areas were identified for inclusion, to broaden the scope of ICCS 
2016: 


+ Environmental sustainability in civic and citizenship education: In many societies, 
the potential impact of human activity on the environment (in particular on the 
global climate) and environmental sustainability have become key issues in debates 
about their future political, social and economic development, which is reflected 
in many international and declarations (see for example UNESCO [United 
Nations Educational, Scientific and Cultural Organization], 2015). Over the past 
decades, responsible citizenship has been increasingly viewed as including regard 
for the environment and its long-term protection, requisite for future sustainable 
development (Dobson, 2003; Dobson, & Bell, 2006; Ferreira, 2013; Hayward, 
2006), and nowadays many education systems put emphasis on the protection of 
the environment or education for environmental sustainability in their citizenship 
curricula (Ainley et al., 2013; Eurydice [Education Information Network in Europe], 
2012; Schulz, et al., 2010b). 


IEA ICCS 2016 — ASSESSMENT FRAMEWORK 


* Social interaction at school: Reviews of civic and citizenship education curricula 
across countries provide evidence that at the outset of the 21st century a large 
number of countries place emphasis on non-formal aspects of civic learning through 
participation and engagement or social interaction at schools (Ainley et al., 2013; 
Eurydice, 2005; Schulz et al., 2010b). More generally, research increasingly recognizes 
the importance of social learning within schools (Dijkstra, & dela Motte, 2014; Durlak, 
Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Scheerens, 2011). Therefore, ICCS 
2016 is designed to include more aspects related to social interaction at school in 
the survey instruments, in particular items related to relationships within the school 
community, including those related to conflict and the use of verbal and physical 
abuse (for example bullying) (Merrell, Gueldner, Ross, & Isava, 2008; Rigby, & Smith, 
2011; Smith, Morita, Junger-Tas, Olweus, Catalano, & Slee, 1998). 


+ The use of new social media for civic engagement: There is growing evidence about 
the importance of new social media! and the use of such media has been found to 
have a profound effect on civic engagement among young people (Anduiza, Jensen, & 
Jorba, 2012; Bachen, Raphael, Lynn, McKee, & Philippi, 2008; Banaji, & Buckingham, 
2013; Kahne, Lee, & Feezell, 2011). Given the further increases in engagement with 
social media and its relevance for communication on social and political issues since 
the previous ICCS survey, it was seen as important to explore the use of new social 
media for civic engagement in greater detail in ICCS 2016. 


In addition, two further areas were identified that had been included in previous IEA 
surveys as deserving more explicit acknowledgement in the ICCS 2016 assessment 
framework: 


e Economic awareness as an aspect of citizenship: Students’ economic awareness 
may be regarded as an important aspect of civic and citizenship education (see for 
example, Citizenship Foundation, 2013; Davies, 2006, 2015; Davies, Howie, Mangan, 
& Telhaj, 2002). It can be conceptualized as a broad awareness of the ways in which 
economic issues influence citizenship (rather than financial or economic literacy’). 
Economic awareness is relevant to civic and citizenship education because economics 
is a major focus of government, economic conditions provide constraints on some 
citizenship activities, citizens contribute to the economic well-being of society, and 
citizens share responsibility for economic problems and remedies. 


* The role of morality in civic and citizenship education: Concepts of morality and 
character are often invoked in relation to outcomes of civic and citizenship education 
programs (Althof, & Berkowitz, 2006; Berkowitz, Althof, & Jones, 2008; Halstead, & 
Pike, 2006; Oser, & Veugelers, 2008). Many countries have moral education programs 
(often integrated with civic and citizenship education) and moral education is also 
often regarded as an independent field of study (Ainley et al., 2013; Veugelers, 2011). 
Therefore, the assessment framework provides scope for explicit representations of 
morality in the ICCS 2016 instruments. 


1 New social media: a collection of online social networking sites and tools (for example, Facebook, Twitter, YouTube) 
and shared content sites (for example, wikis, blogs, discussion forums) that people use to socially interact and distribute 
content with other groups of people. 

2 An assessment of students’ knowledge and perceptions of the economy is difficult given the age of the ICCS 2016 target 
population (aged 13-14). In CIVED, cognitive items measuring economic literacy were only included in the survey of 
upper secondary students aged 16 to 18 (see Amadeo et al., 2002). The OECD Programme for Student Assessment (PISA) 
assessed the financial literacy of 15-year-old students in its 2012 survey cycle (see OECD, 2014a). 
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1.5 


Research questions 


The key research questions for ICCS 2016 concern students’ civic knowledge, their 


dispositions to engage, and their attitudes related to civic and citizenship issues, as well 


as contexts for this learning area. Some of the key research questions are similar to those 


that were formulated for ICCS 2009. Each general research question (RQ) relates to a 


subset of specific research questions to be addressed in ICCS 2016. 


RQ1 


RQ 2 


RQ 3 


How is civic and citizenship education implemented in participating countries? 
This research question is concerned the national contexts for civic and citizenship 
education and includes the following specific research questions: 


(a) What are the aims and principles of civic and citizenship education in each 
participating country? The analyses will be focused on information from the 
national contexts survey with references to published sources (for example, 
national curriculum documents) about the background and intentions 
behind civic and citizenship curricula in participating countries. 


(b) Which curricular approaches do participating countries choose to provide 
civic and citizenship education? The analyses will have a focus on different 
types of civic and citizenship education implemented in participating 
countries and may be based on national contexts survey data, published 
sources and school survey data. 


(c) What changes and/or developments in this learning area can be observed 
since 2009? The analyses will include only data from countries participating 
in both ICCS surveys and focus on reforms and changes in the national 
contexts for civic and citizenship education. 


What is the extent and variation of students’ civic knowledge within and 
across participating countries? Analyses to address this research question would 
primarily focus on student test data supplemented by information collected 
through the student questionnaire in order to answer the following specific 
research questions: 


(a) Are there variations in civic knowledge associated with student characteristics 
and background variables? These analyses would investigate the influence of 
student gender, family characteristics, socioeconomic indicators and other 
background variables on civic knowledge. 


(b) What contextual factors explain variation in students’ civic knowledge? 
Analyses would study the relationship between contextual variables such as 
home background or school characteristics at different levels with variation 
in students’ civic knowledge. 


(c) What changes in civic knowledge have occurred since 2009? These analyses 
would be limited to those countries participating in both ICCS surveys and 
require comparable measures of civic knowledge over time. 


What is the extent of students’ engagement in different spheres of society and 

which factors within or across countries are related to it? This research question 

is related to indicators of student engagement and encompasses the following 

specific research questions: 

(a) What is the extent and variation of students’ civic participation in and out 
of school? The analyses will focus on student reports on past and current 
involvement in civic-related activities. 


RQ4 


RQ5 
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(b) What beliefs do students hold regarding their own capacity to engage and the 
value of civic participation? The analyses will focus on student perceptions 
of civic engagement. 


(c) What expectations do students have regarding civic and political participation 
in the near future or as adults? The analyses will address students’ behavioral 
intentions regarding different forms of civic or political participation. 


(d) What changes in student engagement can be observed since 2009? The 
analyses will include data from those countries participating in both ICCS 
surveys and engagement indicators included in both studies. 


What beliefs do students in participating countries hold regarding important 
civic issues in modern society and what are the factors influencing their 
variation? This research question is related to different student affective measures 
and encompasses the following specific research questions: 


(a) What attitudes do students hold toward civic institutions and society? The 
analyses will address the way students perceive society in general, its rules 
and institutions. 


(b) What are students’ beliefs regarding the importance of different principles 
underlying society? The analyses should focus on students’ beliefs about 
democracy, citizenship and diversity. 


(c) What are students’ perceptions of their communities and societies? The 
analyses will be related to students’ sense of identity within their (local, 
national and supra-national) communities and connections with others in 
society. 

(d) What changes in student beliefs can be observed since 2009? The analyses will 
include only data from those countries participating in both ICCS surveys 
and affective-behavioral measures included in both studies. 


How are schools in the participating countries organized with regard to 
civic and citizenship education and what is its association with students’ 
learning outcomes? This research question is related to ways schools (within 
their community) provide for spaces for civic and citizenship education and 
encompasses the following specific research questions: 


(a) What are the general approaches to civic and citizenship education, 
curriculum, and/or program content structure and delivery? 


(b) To what extent do schools in participating countries have participatory 
processes in place that facilitate civic engagement? The analyses will primarily 
focus on teacher and school survey indicators regarding the school climate 
for civic engagement. 


(c) To what extent do schools and communities interact to foster students’ civic 
engagement and learning? The analyses will primarily focus on teacher and 
school survey indicators regarding the schools’ interactions with their local 
communities and opportunities for students’ active civic involvement. 


(d) How do schools and teachers perceive the role of civic and citizenship 
education across participating countries? The analyses will address how 
teachers, principals and policies perceive the role schools and teacher play in 
preparing young people for citizenship, and to what extent these perceptions 
have changed since ICCS 2009. 
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These research questions played a central role in shaping the design of ICCS 2016 and 
its instrumentation, and in guiding the development of the assessment framework. 
Furthermore, they will provide the basis for organizing the reporting of findings from 
this study. 


1.6 Study design 


Consistent with ICCS 2009, the student population to be surveyed includes students in 
their eighth year of schooling (on average including students who are approximately 14 
years of age). Typically, this will be Grade 8 students, provided that the average age of 
students at this year level is 13.5 years or above. In countries where the average age of 
students in Grade 8 is less than 13.5 years, Grade 9 is defined as the target population. 
In each sampled school, intact classrooms are selected, and all students in a class are 
assessed for the ICCS 2016 survey. 


The definition of the target population of teachers is the same as in the previous ICCS 
teacher survey. It includes all teachers teaching regular school subjects to the students 
in the target grade at each sampled school, but is limited to those teachers teaching the 
target grade during the testing period and employed at school since the beginning of the 
school year (Zuehlke, 2011). Fifteen teachers are randomly selected from each school 
participating in the ICCS survey. There is also an international option to ask teachers 
of civic-related subjects at the target grade additional questions on civic teaching and 
learning. 


An important feature of ICCS 2009 was the establishment of regional modules in Asia, 
Europe and Latin America (Fraillon et al., 2012; Kerr et al., 2010, 2011; Schulz, Ainley, 
Friedman, & Lietz, 2011). The regional modules were composed of groups of countries 
from the same geographic region, which together administered additional instruments 
to assess region-specific aspects of civic and citizenship education. ICCS 2016 includes 
regional instruments for countries in Europe and Latin America. The content of the 
regional instruments focuses on topics that are not covered in the international survey 
material and of particular relevance in the countries of the particular geographic region. 


The following instruments are administered as part of the ICCS survey: 


+ An international cognitive student test consisting of items measuring students’ civic 
knowledge and ability to analyze and reason. 


°- A student questionnaire consisting of items measuring student background variables 
and student perceptions. 


* Regional student instruments consisting of questionnaire-type items. These 
instruments are only administered in countries participating in the European and 
Latin American modules. 


* A teacher questionnaire, administered to selected teachers teaching any subject in the 
target grade. It gathers information about teacher background variables and teachers’ 
perceptions of factors related to the context of civic and citizenship education in 
their respective schools. As in ICCS 2009, participating countries have the option of 
offering an online administration of this questionnaire. 


* A school questionnaire, administered to school principals of selected schools to capture 
school characteristics and school-level variables related to civic and citizenship 
participation. As with the teacher questionnaire, the school questionnaire may be 
completed online in some countries. 
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* The national contexts survey, completed online by national center experts, is designed 
to gather data about the structure of the education systems, the status of civic and 
citizenship education in the national curricula, and recent developments. The data 
obtained from this survey will supplement published information sources about 
countries and their education systems to assist with interpretation of the results from 
the student, school, and teacher questionnaires, and in describing national contexts 
for civic and citizenship education. 


1.7 Characteristics and structure of the assessment framework 


The assessment framework provides a conceptual underpinning for the international 
instrumentation for ICCS 2016 and the development of regional instruments for 
European and Latin American countries. It should be noted that the assessment 
framework is a continuation of the ICCS 2009 framework and that some parts remain 
unchanged. However, while its basic orientation and coverage is consistent with the 
assessment framework for ICCS 2009 in order to ensure continuity across survey cycles, 
it has also been refined as well as modified in order to capture new developments in the 
area of civicand citizenship education. It is designed to have the following characteristics: 


+ It maintains a high degree of consistency with ICCS 2009. 


°- It reflects contemporary research findings on civic and citizenship education among 
students at secondary school (for example, Pancer, 2015). 


* It appropriately addresses the needs and interests of participating countries. 
°- It limits content to aspects that can be adequately measured. 


* Itaims to include all relevant aspects of content that describe the breadth of civic and 
citizenship education across participating countries. 


* It takes the whole range of contexts within which civic and citizenship education 
occurs into account. 


The assessment framework for ICCS 2016 consists of the following three parts: 


* The civic and citizenship framework outlines the aspects to be addressed when 
measuring cognitive and affective-behavioral constructs related to civic and 
citizenship education through the student test and questionnaires. 


* The contextual framework describes the different context factors that might influence 
student learning outcomes related to civic and citizenship education, and which are 
measured through the student, teacher, school and national contexts questionnaires. 


+ The assessment design provides an overview of the ICCS instruments, the coverage 
of framework domains, the different item types, the assessment design, and the 
expected cognitive, affective-behavioral and contextual indices. 
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2 Civic and Citizenship Framework 


2.1 Defining civics and citizenship 


2.1.1 The scope of civics and citizenship in ICCS 2016 


The civic and citizenship framework identifies and defines those aspects of cognitive and 
affective-behavioral content’ that should be considered important learning outcomes 
of civic and citizenship education. It should be noted that within the context of this 
framework the term “learning outcomes” is used in a broad way and is not confined 
to school learning. The way students develop cognitive knowledge and understanding 
of civics and citizenship, as well as affective-behavioral dispositions towards civics and 
citizenship, may depend on many factors beyond their learning environment at school 
(see Amna, Ekström, Kerr, & Stattin, 2009; Pancer, 2015; Pancer, & Pratt, 1999). The 
factors influencing students’ development of these learning outcomes are mapped and 
described in the contextual framework. 


The conception of civic and citizenship education underpinning ICCS 2016 places the 
central focus on the individual student who is influenced by “agents of socialization” in 
varied learning environments (Torney-Purta et al., 2001). It reflects a view that learning 
about civics and citizenship is not limited to instruction in schools but is an outcome 
from a range of processes that take place in different environments. Young people learn 
about civics and citizenship through their interactions with a range of significant others 
and the various communities with which they are associated. This view has continued 
through the predecessors of ICCS 2016: CIVED 1999 and ICCS 2009. It is a view that 
has evolved from ecological systems theories (Bronfenbrenner, 2004; Neal, & Neal, 
2013) and theories of situated cognition (Anderson, Greeno, Reder, & Simon, 2000). 


The field of civic and citizenship education includes cognitive aspects of learning as 
well as the development of attitudes towards aspects of civic life and dispositions to 
participate actively in the life of communities. One of the important contributions of 
the IEA studies investigating civic and citizenship education has been the emphasis on 
the role of cognitive skills in the preparation of young people to fulfill their roles as 
citizens. IEA studies of civic and citizenship education have recognized that, in order to 
participate effectively as citizens, young people need to possess a knowledge base and 
the capacity to reason about the institutions, events, actions and processes that exist in 
their civil and civic communities, as well as to develop and justify views and attitudes 
towards those things. In addition it recognizes that, as students come to know about 
and process cognitive aspects of civics and citizenship, they also develop and refine 
attitudes to, and dispositions to participate in, civic life. Conversely, as they develop 
interests in and a propensity to participate in aspects of civic life so they also learn and 
understand more about key aspects of civics and citizenship. 


3 To describe cognitive and affective-behavioral content in general, we use the term "civics and citizenship" in this 
framework. However, for describing processes, concepts or domains we use "civic and citizenship" (without “s”) in 
conjunction with the corresponding term or object (for example, the "civic and citizenship framework" or "civic and 
citizenship education"). 
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2.1.2 The representation of civics and citizenship in the framework 


The ICCS 2016 Civic and Citizenship Framework is organized around four content 
domains: 


* Content domain 1: Civic society and systems. 
* Content domain 2: Civic principles. 
* Content domain 3: Civic participation. 


* Content domain 4: Civic identities. 


Content domains describe areas of civics and citizenship about which individuals may 
have developed knowledge and understanding, and towards which they may also have 
developed perceptions and dispositions. Therefore, topics included in each content 
domain concern cognitive aspects (i.e. how much students know and understand), and 
perceptions (i.e. how students view them and are willing to act on them). 


Cognitive processes may be viewed as (i) remembering or recalling information or 
processing content in terms of understanding, or (ii) applying an understanding to 
new situations (see Anderson, & Krathwohl, 2001). Similar to the structure of cognitive 
domains in other IEA studies (see for example: Mullis, & Martin, 2013), two following 
cognitive domains are identified in the ICCS 2016 framework: 


* Cognitive domain 1: Knowing. 


* Cognitive domain 2: Reasoning and applying. 


This represents a change in terminology from ICCS 2009 for the Cognitive domain 2 
due to the observation that the terms “reasoning” and “analyzing” were conceptually 
quite similar. Substituting “analyzing” with “applying” is proposed to indicate that 
students should be assessed with regard to their ability to make effective use of their 
civic knowledge in real world contexts. 


Given the central importance of students’ attitudes and dispositions to engage in society, 
the framework includes an affective-behavioral dimension that concerns the attitudes 
that students develop and their disposition to participate in the civic life of their 
societies (see Schulz, Losito, & Kerr, 2011). The framework envisages this dimension 
as consisting of two major affective-behavioral domains that are identified in the ICCS 
assessment framework as: 


* Affective-behavioral domain 1: Attitudes. 


+ Affective-behavioral domain 2: Engagement. 


The two affective-behavioral domains attitudes and value beliefs described in the ICCS 
2009 framework have been consolidated into affective-behavioral domain 1 in 2016. 
This was done to address concerns about whether the implicit distinction between 
more enduring and deeply-rooted attitudes from those that are more focused on 
specific issues and more time-specific was appropriate for adolescents in the age group 
under study. In the ICCS 2016 framework, the ICCS 2009 affective-behavioral domains 
behavioral intentions and behaviors were combined, together with dispositions related to 
civic participation such as interest or self-efficacy, which had been classified as attitudes 
in the previous study, into the affective-behavioral domain 2 (engagement). It needs to 
be recognized, however, that recent or past civic-related activities can also be regarded 
as contextual factors influencing learning outcomes. 
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2.1.3 Mapping assessment domains to the assessment instruments 


The ICCS assessment of the learning outcomes of civic and citizenship education 
comprises two types of instrument: 


°- A student test measuring cognitive representations and processes. 


+ Student questionnaires (international and regional) measuring affective-behavioral 
variables reflecting attitudes and engagement. 


Data from the cognitive test will contribute to further elaboration of the proficiency 
scale of civic and citizenship knowledge established in ICCS 2009. The content of the 
scale is derived from the substance of the four content domains as operated on through 
the two cognitive domains. 


Data from the international and regional student questionnaires will be used to 
articulate constructs pertaining to the two affective-behavioral domains and relating 
to the substance of the four content domains. The amount and type of assessment 
information accessed by each instrument will vary across the four content domains. 


2.2 Civic and citizenship content domains 
2.2.1 Structures and key terms in the ICCS 2016 assessment framework 


Structure of the content domains 


The four content domains of the ICCS 2016 assessment framework share the following 
structures: 


e Sub-domain: This refers to a substantive or contextual component of a content 
domain. The sub-domains are described if they include sufficient discrete content to 
warrant individual definition and articulation. This model anticipates some overlap 
between the sub-domains within each domain. 


+ Aspect: This refers to specific content regarded as largely situated within a given sub- 
domain. 


* Key concept: This refers to concepts and processes common to sub-domains within a 
given content domain. 


In short, each content domain is divided into sub-domains, and each sub-domain 
consists of one or more aspects. The key concepts can be expressed within the contexts 
of any of the sub-domains. Figure 2.1 illustrates the structure of the content domains 
in the ICCS 2016 assessment framework. 
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Figure 2.1: The structure of the ICCS 2016 assessment framework content domains 
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Note: KC = Key concept; Sd = Sub-domain; ASd = Aspect. 


Structure of the affective-behavioral domains 


The two affective-behavioral domains are described in terms of the types of student 
attitudes and engagement indicators relevant with respect to the civic and citizenship 
content domains. 


Structure of the cognitive domains 


The two cognitive domains are defined in terms of the cognitive processes that comprise 
them. This includes the assumption that processes are to be applied to content from 
within the four civic and citizenship content domains. 


Key terms 


The ICCS assessment framework frequently uses a set of key terms. The following 
definitions of these key terms are those used in this framework. The definitions of many 
of the terms used in the framework are the subject of ongoing and vigorous academic 
dialogue. The definitions of the key and domain-specific terms in this framework have 
been constructed to support consistent understandings of the framework’s contents 
across the broad range of countries participating and interested in ICCS. 
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Community A group of people who share something in common (for example, history, 
values, loyalties, acommon goal). In this framework, community membership 
includes membership based on externally defined criteria relating to the 
function of the community (such as attending a school as a student) and 
membership defined by individuals’ own belief of their membership (Such 
as through identification with “like-minded” people regarding a political, 
religious, philosophical or social issue).* 


Society A community defined by its geographical territory and within which the 
population shares a common culture (which may comprise and celebrate 
multiple and diverse ethnic or other communities) and way of life under 
conditions of relative autonomy, independence, and self-sufficiency. 


Citizenship® a. The legal status of being a citizen of a nation state or supranational legal 
community (for example the European Union). 


b. The fact of individuals’ participation, or lack of participation, in their 
communities. The term “citizenship,” unlike the term “active citizenship,” 
does not assume certain levels of participation. 


Civil Refers to the sphere of society in which the shared connections between 
people are at a broader level than that of the extended family but do not 
include connections to the state. 


Civic Refers to any community in which the shared connections between people 
are at a broader level than that of the extended family (including the state). 
Civic also refers to the principles, mechanisms, and processes of decision- 
making, participation, governance, and legislative control that exist in these 
communities. 


2.3 Civic and citizenship content domains 


The first content domain, civic society and systems, comprises the mechanisms, systems, 
and organizations that underpin societies. The second domain, civic principles, refers to 
the shared ethical foundations of civic societies. Civic participation deals with the nature 
of the processes and practices that define and mediate the participation of citizens in 
their civic communities (often referred to as active citizenship). The ICCS assessment 
framework recognizes the centrality of the individual citizen through the fourth content 
domain civic identities. This domain refers to the personal sense an individual has of 
being an agent of civic action with connections to multiple communities. Together, 
these four domains describe the civic and citizenship content to be assessed in ICCS. 


2.3.1 Content domain 1: Civic society and systems 


The content domain Civic society and systems focuses on the formal and informal 
mechanisms and organizations that underpin both the civic contracts that citizens 
have with their societies and the functioning of the societies themselves. The three sub- 
domains of civic society and systems are: 


e Citizens. 
e State institutions 


e Civil institutions. 


4 Note that a community may still contain a level of diversity. 
Note that “Global Citizenship” has been included as a concept in content domain 4 (civic identities). 
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Citizens 


The sub-domain Citizens focuses on the civic relationships between individuals and 
groups of citizens and their societies. The aspects of this sub-domain relate to knowledge 
and understanding of as well as beliefs about: 


* Citizens’ and groups’ assigned and desired roles within their civic society 
* Citizens’ and groups assigned and desired rights within their civic society 
e Citizens’ and groups’ assigned and desired responsibilities within their civic society 


e Citizens’ and groups’ opportunities and abilities to engage within their civic society. 


State institutions 

The sub-domain State institutions focuses on those institutions central to the processes 
and enacting of civic governance and legislation in the common interest of the people 
they represent and serve. 

The aspects of this sub-domain are: 

+ Legislatures/parliaments 

* Governments 

* Economic structures, mechanisms and conditions 

+ Supranational/intergovernmental governance bodies 

* Judiciaries 

e Law enforcement bodies 

e National defense forces 

e Bureaucracies (civil or public services) 


* Electoral commissions. 


Civil institutions 

The sub-domain Civil institutions focuses on those institutions that can mediate 

citizens’ contact with their state institutions and allow citizens to actively pursue many 

of their roles in their societies. 

The aspects of this sub-domain are: 

e Companies/corporations 

+ Trade unions 

e Political parties 

+ Non-governmental organizations (NGOs) 

+ Advocacy groups (for example, pressure, lobby, campaign, special interest groups) 

° Traditional media (for example, newspaper, television and radio) 

e New social media (for example, web forums, blogs, twitter, Facebook, and text 
messaging) 

* Religious institutions 

* Schools 


* Cultural organizations. 
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Key concepts 


Power/authority 


Rules/law 


Constitution 


Governance 


Decision-making 
Negotiation 


Accountability 


Democracy 


Sovereignty 


Nation-building 


Statelessness 


Listed together as concepts dealing with the nature and consequences 
of the right or capacity of bodies or individuals to make binding decisions 
on behalf of others that these others are then required to accept and to 
adhere to. 


Listed together as the explicit and implicit prescriptions for behavior. 
Rules are those prescriptions that are not required to be and are 
therefore not enforced by a sovereign body. Laws are considered to be 
those prescriptions that are enforced by a sovereign body. 


The fundamental rules or laws of principle governing the politics of a 
nation or sub-national body. 


The act and the processes of administering public policy and affairs. 


The formal and informal processes by which decisions are made within 
and among civil and state institutions. 


The processes that underpin and are evident in negotiation, and the use 
and necessity of negotiation as a means of decision-making. 


The requirement for representatives to answer to those they represent 
about the representatives’ conduct of their duties and use of their 
powers. Accountability includes the assumption that representatives 
are able to accept responsibility for their failures and to take action to 
rectify them. 


The ICCS assessment framework accepts the broadest definition 
of democracy “as rule by the people.” This definition refers both 
to democracy as a system of governance and to the principles of 
freedom, equity, and sense of community® that underpin democratic 
systems and guarantee respect for and promotion of human rights. 
Both representative democratic systems (such as national parliaments) 
and “direct democracy” systems (such as through referendums or 
systems used in some local community or school organizations) can 
be examined as democratic systems under the definition of democracy 
used in this framework. 


The claim of each individual state/nation to have the ultimate power in 
making political decisions relevant to that state/nation and recognition 
that this power underpins the operation and viability of international 
organizations, agreements, and treaties. 


The process of developing among the people of a nation some form of 
a unified sense of national identity, with the aim of fostering long-term 
harmony and stability. Within the parameters of the ICCS assessment 
framework, nation-building is assumed to be a dynamic ongoing 
process in all nations rather than a process associated only with newly 
independent nations. 


The circumstances of people who do not have any legal bond of 
nationality or citizenship with any state. Included in this concept are the 
causes and consequences of statelessness. 


6 See Civics and Citizenship Content Domain 2. 
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Franchise/ Listed together, these concepts refer to the rights, responsibilities, 

Voting and expectations of people to vote in formal and informal settings. 
These concepts also refer, more broadly, to issues associated with 
voting and voting processes, such as compulsory and voluntary voting 
and secret ballots. 


The economy Systems governing the production, distribution, and consumption 
of goods and services within states, including industrial regulation, 
trade, taxation, and government spending including on social welfare. 
Economic conditions are both a focus of civic decisions as well as a key 
aspect of the environment in which decisions about other policies are 
made. 


The welfare state The role of a government in providing for the social and economic 
security of its people through support such as health care, pensions, 
and social welfare payments and benefits. 


Treaties Binding agreements under international law entered into by eligible 
bodies such as states and international organizations. 

Sustainable Development that meets the needs of the present without 

development compromising the ability of future generations to meet their own needs. 


Factors that can be considered in terms of sustainable development 
include environmental protection, economic development, social 
equality, and social justice. 


Environmental A state in which demands placed on the natural world can be met 
sustainability without negatively impacting on the natural world or reducing its 
capacity to support human life. 


Globalization The increasing international movement of commodities, money, 
information, and people; and the development of technology, 
organizations, legal systems, and infrastructures to allow this 
movement. The ICCS assessment framework acknowledges that a 
high level of international debate surrounds the definition, perceptions, 
and even the existence of globalization. Globalization has been included 
in the framework as a key concept for consideration by students. The 
definition is not a statement of belief about the existence or merits of 
globalization. 


Dissent In democratic societies, dissent is a central notion that allows for 
voicing opposition to, expressing disagreement with, or standing apart 
from, the policies or decisions of the governing body. 


2.3.2 Content domain 2: Civic principles 


The content domain ‘Civic principles focuses on the shared ethical foundations of civic 
societies. The framework regards support, protection, and promotion of these principles 
as civic responsibilities and as frequently occurring motivations for civic participation 
by individuals and groups. The domain consists of four sub-domains: 


+ Equity 
* Freedom 
* Sense of community 


* Rule of law. 
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Equity 

The sub-domain Equity focuses on the principle that all people have the right to fair 
and just treatment, and that protecting and promoting equity is essential to achieving 
peace, harmony, and productivity within and among communities. The principle of 
equity is derived from the notion of equality—that all people are born equal in terms 
of dignity and rights. 


Freedom 


The sub-domain Freedom focuses on the concept that all people should have freedom 
of belief, freedom of speech, freedom from fear, and freedom from want, as articulated 
in the United Nations Universal Declaration of Human Rights (United Nations, 1948). 
Societies have a responsibility to actively protect the freedom of their members and to 
support the protection of freedom in all communities, including those that are not their 
own. 


Sense of community 


The sub-domain Sense of community is related to the sense of belonging and 
connectedness within societies, and focuses on collective responsibility and common 
vision that exists amongst the individuals and communities within a society. When a 
strong sense of community exists individuals actively appreciate and acknowledge the 
diversity of individuals and communities that comprise a society as well as demonstrate 
responsibility toward its development. It is acknowledged (in regard to this sub- 
domain) that manifestations of sense of community vary between societies, that there 
may be tensions within societies between demands for social cohesion and the existing 
diversity of views and actions, and that the resolution of these tensions is an ongoing 
area of debate within many societies. 


Rule of law 


The sub-domain Rule of law is related to the principle that all citizens, institutions 
and entities including the State itself are subject and accountable to laws, which are 
publicly promulgated, independently adjudicated, equally enforced and consistent with 
international standards and norms protecting human rights (United Nations, 1948). It 
requires recognition of the supremacy of law, the concept that all citizens are equal before 
the law regardless of their background and personal characteristics (such as gender, 
race, religion), fairness in the application of law, separation of powers, participation in 
decision-making, legal certainty, as well as legal and procedural transparency. 


Key concepts 


Concern for the The concept that the ultimate goal of civic and community action is to 
common good promote conditions that advantage all members of the community. 


Human rights A form of inalienable entitlement of all human beings that, for the purpose of 
the ICCS assessment framework, is framed by the contents of the United 
Nations Universal Declaration of Human Rights (United Nations, 1948). 


Empathy Intellectually or emotionally taking the role or perspective of others. 


Social justice The distribution of advantage and disadvantage within communities. 
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Inclusiveness The concept that communities have a responsibility to act in ways that 
support all their members to feel valued as members of those communities. 


Equality The notion that all people are born equal in terms of dignity and rights 
regardless of their personal characteristics (such as gender, race, religion). 


Separation The concept that three branches of government (executive, legislative, 

of powers judicial) are kept separate (independent) from each other to prevent abuse 
of power and establish a systems of checks and balances between these 
branches. 


2.3.3 Content domain 3: Civic participation 


The content domain Civic participation refers to the manifestations of individuals’ 
actions in their communities. Civic participation can operate at any level of community 
and in any community context (including schools as the imminent context for the 
age group under study). The level of participation can range from awareness through 
engagement to influence. The three sub-domains of civic participation are: 


e Decision-making 
°- Influencing 


e Community participation. 


Decision-making 

The sub-domain Decision-making focuses on active participation that directly results 
in the implementation of policy or practice regarding the individual’s community or a 
group within that community. The aspects of this sub-domain are: 


e Engaging in organizational governance 


e Voting. 


Influencing 


The sub-domain Influencing focuses on actions aimed at informing and affecting 
any or all of the policies, practices, and attitudes of others or groups of others in the 
individual’s community. The aspects of this sub-domain are: 


+ Engaging in public debate (including participation through social media) 


+ Engaging in demonstrations of public support or protest (including “virtual” 
engagement through the use of, for example, online petitions) 


+ Engaging in policy development 
* Developing proposals for action or advocacy 


* Selective purchasing of products according to ethical beliefs about the way they were 
produced (ethical consumption/ethical consumerism) 


e Recognizing corruption. 


Community participation 

The sub-domain Community participation focuses on participation, with a primary 
focus on enhancing a person’s connections with a community, for the ultimate benefit 
of that community. The aspects of this sub-domain are: 
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* Volunteering 
e Participating in organizations 


e Acquisition of information. 


Key concepts 


Civic The concept that civic communities benefit from the active 

engagement engagement of their citizens and that therefore civic communities have 
a responsibility to facilitate active citizenship and that citizens have a 
responsibility to participate actively in their civic communities. 


Co-operation’ The concept that communities benefit most when their members 

collaboration act together in pursuing the common goals of the community. This definition 
allows for disagreement within communities about the best way to achieve 
their goals. 


Negotiation/ The concept that peaceful resolution of differences is essential 
resolution to community well-being and that negotiation is the best way to attempt to 
reach resolutions. 


Engagement The concept that citizens need to concern themselves with issues and 
information in their communities in order to participate effectively. 


2.3.4 Content domain 4: Civic identities 


The content domain Civic Identities includes the individual’s civic roles and perceptions 
of these roles. As was the case with the conceptual model of CIVED, ICCS assumes 
that individuals both influence, and are influenced by, the relationships they have with 
family, peers, and civic communities. Thus, an individual’s civic identity explicitly links 
to a range of personal and civic interrelationships. This framework asserts and assumes 
that individuals have multiple articulated identities rather than a single civic identity. 
Civic communities include points of reference at many levels ranging from family and 
local community to geographical regions or the global community. 


Civic identities comprises two sub-domains: 
* Civic self-image 


* Civic connectedness. 


Civic self-image 

The sub-domain Civic self-image refers to the individual’s experience of their place in 
each of their civic communities. Civic self-image focuses on the individual’s civic and 
citizenship values and roles, the individual’s understanding of and attitudes toward 
these values and roles, and the individual’s management of these values and roles 
whether they are in harmony or in conflict within the individual. 


Civic connectedness 

The sub-domain Civic connectedness refers to the individual’s sense of connection to 
their different civic communities and to the different civic roles the individual plays 
within each community. 
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Civic connectedness also includes the individual’s beliefs about and tolerance of the 
levels of diversity (of civic ideas and actions) within and across their communities, and 
their recognition and understanding of the effects of the range of civic and citizenship 
values and belief systems of their different communities on the members of those 
communities. 


Key concepts 


Civic Individuals’ view of themselves as citizens in their civic communities. 

self-concept This view includes individuals’ sense of the communities to which they 
belong and their capacity to identify the nature and parameters of their civic 
roles in their communities. 


Multiplicity Individuals’ sense of the range of different roles and potentials they have 
within and across their different communities. Included in this concept is the 
understanding that an individual’s roles and potentials are ever changing and 
that these depend on the context of each separate community connection. 


Diversity Individuals’ sense of and level of acceptance of the range of people and 
viewpoints that exist within and across their communities. 

Cultures/ Individuals’ sense of the value and place of the cultures they associate 

location with their communities in their own civic lives and the civic lives of the other 


members of their communities. 


Patriotism An individual’s love for or devotion to their country (or countries), which can 
lead to a willingness to act in support of that country (or countries). 


Nationalism The politicization of patriotism into principles or programs based on the 
premise that national identity holds precedence over other social and 
political principles. 


Global Individuals’ sense of belonging to the global community and common 
citizenship humanity that transcends local and national boundaries. 

Civic and Individuals’ central ethical and moral beliefs about their civic communities 
citizenship communities and their roles as citizens within their communities. 

values 


2.4 Civic and citizenship cognitive domains 


Each of the four content domains encompasses different types of knowledge concerned 
with civics and citizenship (factual, procedural, conceptual and meta-cognitive) 
(Anderson, & Krathwohl, 2001). The framework then considers the extent to which 
students develop the capacity to process the content of the four domains and reach 
conclusions that are broader than any single piece of knowledge. This includes the 
processes involved in understanding complex sets of factors influencing civic actions 
and planning for and evaluating strategic solutions and outcomes. It extends from direct 
applications of knowledge to reach conclusions about concrete situations through to 
the selection and assimilation of knowledge and understanding of multiple concepts 
in order to reach conclusions about complex, multifaceted, unfamiliar and abstract 
situations. This is a simplification of the hierarchy of cognitive processes articulated by 
Anderson and Krathwohl (2001): remembering or recalling information or processing 
content in terms of understanding or applying an understanding to new situations’. 


7 The simplification is intended to reflect what is appropriate for students in the target grade and what is most relevant to 
civics and citizenship. 
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Students’ knowledge about civics and citizenship is measured by the ICCS cognitive 
test. To respond to the test items, students need to know the civic and citizenship 
content being assessed. Students also need to be able to apply more complex cognitive 
processing to their civic and citizenship knowledge and to relate their knowledge and 
understandings to real-world civic action. 


The two ICCS cognitive domains summarize the cognitive processes that students are 
expected to demonstrate in the ICCS cognitive test. The data derived from the test 
items constructed to represent the processes in the cognitive domains will be used to 
construct a global scale of civic and citizenship knowledge and understandings of the 
four content domains. The first cognitive domain, knowing, outlines the types of civic 
and citizenship information that students are required to demonstrate knowledge of. 
The second domain, reasoning and applying, details the cognitive processes that students 
require to reach conclusions. 


2.4.1 Cognitive domain 1: Knowing 


Knowing refers to the learned civic and citizenship information that students use when 
engaging in the more complex cognitive tasks that help them make sense of their civic 
worlds. Students are expected to remember, recall or recognize definitions, descriptions, 
and the key properties of civic and citizenship concepts and content, and to illustrate 
these with examples. Because ICCS 2016 is an international study, the concrete and 
abstract concepts students are expected to know in the core cognitive assessment are 
those that can be generalized across societies. 


Processes 
Define Identify statements that define civic and citizenship concepts and content. 
Describe Identify statements that describe the key characteristics of civic and 


citizenship concepts and content. 


Illustrate with Identify examples that support or clarify statements about civic and 
examples citizenship concepts and content. 


2.4.2 Cognitive domain 2: Reasoning and applying 


Reasoning and applying refers to the ways in which students use civic and citizenship 
information to reach conclusions that are broader than the contents of any single concept 
and to make use of these in real-world contexts. Reasoning and applying includes, for 
example: the use of knowledge to reach conclusions about familiar concrete situations; 
the selection and assimilation of knowledge and understanding of multiple concepts; 
the evaluation of proposed and enacted courses of action; providing recommendations 
for solutions or courses of action. 


Processes 
Interpret Identify statements about information presented in textual, graphical, 
information and/or tabular form that make sense of the information in the light of a civic 
and citizenship concept. 
Relate Use the key defining aspects of a civic and citizenship concept to explain 


or recognize how an example illustrates a concept. 
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Justify Use evidence and civic and citizenship concepts to construct or recognize 
a reasoned argument to support a point of view. 

Integrate Identify connections between different concepts across themes and across 
civic and citizenship content domains. 

Generalize Identify civic and citizenship conceptual principles manifested as specific 
examples and explain how these may apply in other civic and citizenship 
contexts. 

Evaluate Identify judgments about the advantages and disadvantages of alternative 
points of view or approaches to civic and citizenship concepts and actions. 

Suggest Identify courses of action or thought that can be used to alleviate 

solutions civic and citizenship problems expressed as conflict, tension, and/or 


unresolved or contested ideas. 


Predict Identify likely outcomes of given civic and citizenship policies strategies 
and/or actions. 


2.5 Civic and citizenship affective-behavioral domains 


Individuals may have developed beliefs, perceptions, dispositions, behavioral 
intentions and behaviors, which the ICCS 2016 framework conceptualizes as related 
to two affective-behavioral domains: attitudes and engagement. The ICCS student 
questionnaire includes items measuring the affective-behavioral domains that do not 
require correct or incorrect responses and are often measured through use of a Likert- 
type item format in the ICCS 2016 student questionnaires, indicating for example 
the extent to which respondents agree or disagree with a given statement. While most 
constructs or aspects are measured as part of the international student questionnaire, 
others are included as international options or in the regional student questionnaires 
for Europe and Latin America. 


2.5.1 Affective-behavioral domain 1: Attitudes 


The affective-behavioral domain Attitudes refers to judgments or evaluations regarding 
ideas, persons, objects, events, situations, and/or relationships. It is possible for 
individuals to harbor contradictory attitudes at the same time. Attitudes encompass 
responses that are focused on specifics and can change over time, as well as those 
reflecting broader and more fundamental (or deeply rooted) beliefs that tend to be 
constant over longer periods of time.* 


The different types of attitude assessed in ICCS 2016 can be classified depending on 
their (primary) location in the four content domains: 

* Students’ attitudes toward civic society and systems 

* Students’ attitudes toward civic principles 

* Students’ attitudes toward civic participation 


* Students’ attitudes toward civic identities. 


8 More enduring beliefs were conceptualized as “value beliefs” in the ICCS 2009 framework, as opposed to (less enduring 
and more changeable) “attitudes”. However, they now form part of the same affective-behavioral domain in the ICCS 2016 
assessment framework. 
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Students’ attitudes toward civic society and systems 


The following constructs reflecting student attitudes toward civic society and systems 
will be measured using the international student questionnaire and the regional 
European and Latin American questionnaires in ICCS 2009: 


* Students’ perceptions of good citizenship 

e Students’ trust in institutions 

* Students’ perceptions of threats to the world’s future 

* Students’ attitudes toward the influence of religion in society (international option) 
* Students’ perceptions of European future (European regional questionnaire) 

* Students’ attitudes toward European cooperation (European regional questionnaire) 
* Students’ attitudes toward the European Union (European regional questionnaire) 


+ Students’ attitudes toward authoritarian government practices (Latin American 
regional questionnaire). 


Students’ perceptions of good citizenship: This construct refers to student beliefs regarding 
“good citizenship” and relates mainly to Content domain 1 (civic society and systems) 
but also to Content domain 2 (civic principles). Items asking about the importance of 
certain behaviors for “good citizenship” were included in the first IEA study on civic 
education in 1971 (Torney et al., 1975). In CIVED, a set of 15 items asked students to 
rate the importance of certain behaviors for being a good citizen (see Torney-Purta et 
al., 2001, p. 77f). Sub-scales concerned with conventional and with social-movement- 
related citizenship were reported (see Schulz, 2004). Kennedy (2006) distinguished 
active (conventional and social-movement-related) from passive citizenship elements 
(national identity, patriotism, and loyalty). ICCS 2009 included 12 items on good 
citizenship behavior, most of which were similar to those used in CIVED and were used 
to form two scales on conventional and social-movement-related citizenship (Schulz 
et al., 2010b; Schulz, & Friedman, 2011). For ICCS 2016 additional items will measure 
more passive forms of citizenship behavior. 


Students’ trust in institutions: This construct reflects students’ feelings of trust in a 
variety of state and civic institutions in society, and relates mainly to Content domain 
1 (civic society and systems). The first IEA civic education study included one item 
on trust in government (Torney et al., 1975). CIVED used a set of 12 items covering 
political/civic institutions, media, United Nations, schools, and people in general. 
ICCS 2009 used a similar range of 11 core items in a modified format together with 
three optional items on European institutions and state/provincial institutions. Across 
countries, results showed that students tended to express the lowest levels of trust in 
political parties and the highest levels of trust in courts of justice (Schulz et al., 2010b, 
pp. 103-109). In countries with relatively high scores on indices of corruption, and low 
scores on indices of government efficiency, more knowledgeable students expressed less 
trust in civic institutions, whereas positive correlations between civic knowledge and 
trust were recorded in countries with low indices of corruption (Lauglo, 2013). In ICCS 
2016 student trust is measured with the same item set as in ICCS 2009, augmented by 
an item measuring trust in social media. 


Students’ perceptions of threats to the world’s future: It has been reported that students 
express concern about global issues including those regarding poverty, hunger, wars, 
overpopulation and the environment (Holden 2007; Oscarsson, 1996; Rubin, 2002). 
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In ICCS 2016, students are asked to rate the seriousness of a broad range of threats 
to key aspects of civilization such as the extent of poverty, living standards, human 
dignity, economic well-being, and environmental health indicating their personal level 
of concern. These aspects are also reflected in writings concerned with global education 
that envisages broadening student perspectives beyond national contexts (Burnouf, 
2004; Hicks, 2003). Overall these ratings provide an indication of students’ awareness 
of global issues and responses to individual items provide a perspective on profiles of 
concern. 


Students’ attitudes toward the influence of religion in society: Religion is often regarded 
as an important catalyst of civic participation (see Pancer, 2015; Putnam, & Campbell, 
2010; Verba, Schlozman, & Brady, 1995). Smidt (1999) suggested that in the United 
States and Canada religious tradition and church attendance were associated with civil 
participation, even after controlling for the effects of other factors generally associated 
with civic participation. Similar findings have been reported for the United Kingdom 
(Storm, 2015). ICCS 2009 used a set of six items to assess students’ attitudes toward 
religion. The set of items was part of an international option on religious denomination, 
practices, and attitudes toward the influence of religion in society. The results showed 
that in most countries students who attended religious services also held more positive 
attitudes towards the desirability of religious influence on society (Schulz et al., 2010b, 
pp. 107-113). ICCS 2016 includes a slightly modified set of questions regarding religion 
as an international option. 


Students’ perceptions of European future: Recent opinion surveys among European 
citizens have shown that majorities expect that their children’s life will be more difficult 
than theirs, and that Europe’s influence will be weakened in comparison with the 
influence of China or the United States (European Commission, 2014). The ICCS 2016 
European regional questionnaire contains a new question with possible scenarios for 
the European future, asking students to rate the extent of their likelihood of occurring. 


Students’ attitudes toward European cooperation: Recent opinion polls have indicated 
that, in spite of a general surge in anti-European sentiment in some member countries, 
majorities among Europeans support decision-making about important issues at 
the European level (European Commission, 2014). In addition to this, results from 
Standard Eurobarometer survey showed that European citizens consider immigration 
as one of the major challenges that the EU is facing, and that it should be addressed 
through member states cooperation (European Commission, 2015). The European 
regional questionnaire in ICCS 2009 included a question measuring students’ 
perception of harmonization in the European context, and results showed high levels of 
agreement with common European policies (Kerr et al., 2010). The European regional 
questionnaire includes a new question designed to measure students’ endorsement of 
cooperation between European countries regarding a range of different issues. 


Students’ attitudes toward the European Union: Younger people have been reported to 
havea stronger identification with European citizenship than older age groups (European 
Commission, 2013). The European regional survey of ICCS 2009 showed that support 
for the establishment of centralized European institutions was not particularly strong, 
and that support for further enlargement varied considerably across participating 
countries (see Kerr et al., 2010). The European regional questionnaire for ICCS 2016 
includes a question containing statements about the EU designed to measure students’ 
attitudes toward this institution. 
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Students’ attitudes toward authoritarian government practices in Latin America: Surveys 
in the Latin American region have shown considerable support for authoritarian 
government practices among adults and adolescents, and that majorities among adult 
citizens supported non-democratic governments if they solved economic problems 
(United Nations Development Programme, 2004). Support for non-democratic 
government has been shown to be related to educational background, with more 
educated citizens being less supportive of authoritarian government practices 
(Cox, 2010). The Latin American regional questionnaire ICCS 2009 included items 
measuring the endorsement of authoritarian government practices and the justification 
of dictatorships (see Schulz, Ainley, Friedman, & Lietz, 2011). Results showed that 
considerable proportions of lower secondary students in all participating countries 
showed support for non-democratic government practices, and that majorities saw 
dictatorships justified in case they provided economic benefits or more security. The 
Latin American regional questionnaire includes the same item set to measure students’ 
attitudes toward authoritarian government practices and justification for dictatorships. 


Students' attitudes toward civic principles 


The following constructs reflecting student attitudes toward civic principles are 
measured as part of the international student questionnaire, the regional European and 
the Latin American regional questionnaires in ICCS 2016: 


* Students’ attitudes towards democratic values 
+ Students’ attitudes toward gender rights 
* Students’ attitudes toward equal rights for all ethnic/racial groups 


* Students’ attitudes toward equal rights for immigrants (European regional 
questionnaires) 


* Students’ perception of discrimination in European societies (European regional 
questionnaire) 


+ Students views on age limitations for young people (European regional questionnaire) 


* Students’ perception of discrimination of minorities in Latin American societies 
(Latin American regional questionnaire) 


+ Students’ attitudes toward disobedience to the law (Latin American regional 
questionnaire) 


+ Students’ sense of empathy (Latin American regional questionnaire) 


* Students’ attitudes toward homosexuality (Latin American regional questionnaire). 


Students’ perceptions of democracy: This construct refers to student beliefs about 
democracy and relates mainly to Content domain 2 (civic principles). In the IEA 
CIVED survey, students were asked to rate a number of characteristics of society as 
either “good or bad for democracy” (see Torney-Purta et al., 2001). ICCS 2009 used a 
set of nine items that measured the extent of student agreement as to what a society 
should be like using a set of items that were adapted from a subset of those included 
in CIVED. In addition, three items measured students’ beliefs about what should be 
done in response to groups that pose threats to national security. Most of these items 
were endorsed by very large majorities of students across all participating countries 
(Schulz et al., 2010b). In ICCS 2016, students’ attitudes toward democratic values are 
assessed using a different format that requires students to rate a number of possible 
characteristics of a society as “good”, “bad” or “neither good nor bad” for democracy. 
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Students’ attitudes toward gender rights: This construct reflects student beliefs about 
rights for different gender groups in society. The first IEA civic education study in 1971 
included four items measuring support for women’s political rights. The CIVED survey 
used a set of six items to capture students’ attitudes regarding women’s political rights 
(Torney-Purta et al., 2001). ICCS included seven items on gender rights, some of them 
identical with or similar to those used in CIVED. The results showed large majorities 
agreeing with the positive and disagreeing with negative statements about gender 
equity; female students expressed more support for gender equity than males (Schulz 
et al., 2010b, pp. 95-98). The ICCS 2016 student questionnaire includes the same set of 
seven items to measure student attitudes toward gender equality. 


Students’ attitudes toward equal rights for all ethnic/racial groups: This construct reflects 
students’ beliefs about equal rights for all ethnic/racial groups in a country. CIVED 
measured this construct with four items, while ICCS 2009 used five statements to derive 
a scale reflecting attitudes toward equal rights for all ethnic/racial groups (Schulz et al., 
2010b; Torney-Purta et al., 2001). ICCS 2016 uses the same set of items to measure this 
construct. 


Students’ attitudes toward the rights of immigrants in European societies: This construct 
reflects students’ beliefs about rights for immigrants. CIVED measured this construct 
with eight items, five of which were included in a scale reflecting attitudes toward 
immigrants (Schulz, 2004; Torney-Purta et al., 2001). ICCS 2009 included a slightly 
modified version of the same five items used for scaling, together with one additional 
item. In ICCS 2009, students tended to be overwhelmingly in favor of equal rights for 
immigrants with female students being more supportive than males (Schulz et al., 
2010b, pp. 99-102). Results from the European Social Survey among adults suggest 
that public attitudes towards immigration are closely linked to people’s educational 
background (Masso, 2009; Paas, & Halapuu, 2012). Some studies show an increase in 
anti-immigrant attitudes among European youth (Rustenbach, 2010) and, even though 
no systematic data have yet been collected on this issue, further growth in refugee intake 
from the Middle East may have resulted in further changes.’ The regional European 
questionnaire in ICCS 2016 uses the same set of items to measure students’ attitudes 
toward rights of immigrants in their country of residence. 


Students’ views on age limitations for young people in European societies: Legal age limits 
for rights to undertake different activities vary considerably across countries, even 
within the context of Europe. While the minimum voting age (in particular for local 
elections) has been lowered in some European countries, in most countries young 
people can only vote once they are 18. There are also many differences across European 
countries with regard to the required minimum age for buying alcohol; adult surveys 
have shown majorities across European countries supporting 18 as the legal age limit 
for the acquisition of alcohol (European Commission, 2010). The European regional 
questionnaire includes a question asking students to choose between different legal age 
limits that they prefer for a number of different behaviors, including buying alcohol, 
voting, and driving licenses. 


9 It should be noted that perceptions of refugees and immigration may differ, for example, people with positive attitudes 
toward immigration in general may have less favorable views about accepting refugees and vice versa. The recent growth 
in the numbers of refugees in many European countries is not reflected in the development of the ICCS 2016 study and 
the instruments do not address this distinction. 
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Students’ attitudes towards freedom of movement for European citizens within Europe: 
Freedom of movement for European citizens across EU member countries was an 
essential part of the Lisbon Strategy (Bongardt, & Torres, 2012). A recent survey of 
adults within Europe showed that just under half of all respondents were worried 
about immigration from within the European Union (German Marshall Fund, 2014). 
EU member countries tend to have the highest share of free-movement flows in total 
permanent migration movements (OECD, 2012). A high degree of free movement 
of workers is also found in the member countries of the European Economic Area 
(EEA), Iceland, Liechtenstein, and Norway (Alsos, & Eldring, 2008; Dolvik, & Eldring, 
2008). Main challenges to the principle of free movement of persons involve a 
disproportionate monitoring and surveillance of movement of all individuals, together 
with other hidden, as well as visible barriers to make movement and residence more 
inclusive (Carrera, 2005). The European regional questionnaire in ICCS 2009 included 
a set of items measuring students’ perceptions regarding the freedom of movement 
between EU countries for European citizens, which were used to derive two constructs, 
one reflecting support freedom of movement, and the other preferences for restrictions. 
ICCS 2009 results showed student recognition of the benefits of free movement, but 
also large proportions in favor of restricting the movement of workers across borders 
(see Kerr, Sturman, Schulz, & Burge, 2010, pp. 94-98). This principle may come into 
greater prominence in public discussions with the advent of large numbers of refugees 
and displaced people moving to and across Europe. The ICCS 2016 European regional 
questionnaire includes a modified set of six items measuring students’ attitudes toward 
freedom of movement. 


Students’ perceptions of discrimination in European societies: Eurobarometer surveys 
among adults have shown that people perceive quite high levels of discrimination 
across European countries, in particular with regard to the ethnic origin of individuals 
(European Commission, 2012a). It has been suggested that people from European 
countries with more effective antidiscrimination laws are more knowledgeable about 
rights regarding discrimination (Ziller, 2014). The ICCS 2016 European regional 
questionnaire includes a new question presenting students with a number of statements 
about discrimination, for which they should rate their agreement or disagreement. 


Students’ perceptions of discrimination in Latin American societies: Opinion surveys 
among adults across Latin American countries have shown that poor people were 
perceived as suffering most from discrimination, followed by indigenous and afro- 
descendants (Chong, & Nopo, 2007; Nopo, Chong, & Moro, 2010) and that perceptions 
of discrimination were influenced by individuals’ background (skin color and ethnicity), 
as well as contextual factors (Canache, Hayes, Mondak, & Seligson, 2014). The ICCS 
2016 regional questionnaire for Latin America includes a set of items measuring the 
extent to which students perceive social groups as discriminated in their countries. 


Students’ attitudes toward disobedience to the law in Latin America: Cross-national adult 
surveys in the Latin American countries have shown a high level of ambiguity with 
regard to civic morality (i.e. moral behavior and acceptance of disobedience to the 
law), with some countries of the region recording high proportions of acceptance of 
law-breaking (Letki, 2006), which were particular high among young people (Torgler, 
& Valev, 2004). The Latin American regional questionnaire of ICCS 2009 included 
a set of items measuring students’ acceptance of breaking the law under different 
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circumstances, and results showed that larger proportions of young people in the 
participating countries endorsed disobediences to the law, in particular in cases where 
it was perceived as the only way to achieve things, help the family, or when it was done 
without bad intentions. The regional questionnaire for participating Latin American 
countries in ICCS 2016 includes the same item set as in the previous survey, which will 
allow comparisons over time. 


Students’ sense of empathy in Latin America: A sense of empathy relates to the disposition 
of an individual to enter someone else’s world without being influenced by their own 
views and values (Rogers, 1975). Distinctions are made between affective and emotional 
components (Eisenberg, 1995; Strayer, 1987), and cognitive processes that allow people 
to imagine assuming other people’s roles (Piaget, 1965). A sense of emotional empathy 
is viewed as motivating help for others and indicating compassion and concern for 
other human beings (Hoffman, 1981). The Latin American regional questionnaire 
of ICCS 2009 included a question that asked students to rate their level of concerns 
when observing suffering of classmates in different situations. The results showed 
that females tended to express higher levels of empathy than males (Schulz, Ainley, 
Friedman, & Lietz, 2011). The Latin American regional questionnaire for ICCS 2016 
includes the same items as in the previous survey with on additional item designed to 
measure students’ sense of empathy. 


Students’ attitudes toward homosexuality in Latin America: Survey data from the Latin 
American region suggest considerable differences across countries and a divided public 
opinion with regard to attitudes toward homosexuality in the region. For example, 
results from the Latinobarometer show, as a whole, people from the region did not 
feel homosexuality was justified and did not support same sex marriage, although 
considerable country differences were observed (Latinobarómetro, 2009). Attitudes 
towards homosexuality in the region are often influenced by age, gender, education, 
and religious beliefs (Kelley, 2001; Pew Research Center, 2014). In ICCS 2009, the Latin 
American regional questionnaire asked students to rate their agreement with a set of 
positive and negative statements about homosexuality, which did not form a reliable 
scale. In accordance with previous survey research among adults, the results showed 
considerable variation in attitudes across the participating countries, with majorities of 
students in Chile and Mexico supporting the legalization of gay marriage (Schulz, Ainley, 
Friedman, & Lietz, 2011). The regional instrument for Latin America in ICCS 2016 
includes a modified set of items measuring students’ attitudes toward homosexuality. 


Students’ attitudes toward civic participation 


The international and regional questionnaires for Europe and Latin American in ICCS 
2016 include measures regarding the following attitudes related to civic participation: 


* Students’ assessment of the value of student participation at school 
* Students’ attitudes toward political consumerism (European regional questionnaire) 
* Students’ attitudes toward corrupt practices (Latin American regional questionnaire) 


* Students’ attitudes toward violence (Latin American regional questionnaire). 


Students’ assessment of the value of student participation at school: This construct reflects 
students’ beliefs regarding the usefulness of participating in civic-related activities at 
school and is as such closely related to the more general concept of political efficacy. 
Adolescents are generally unable to vote or run for office in “adult politics,” but they 
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experiment as students to determine what degree of power they have to influence the 
ways schools are run (Bandura, 1997, p. 491). CIVED included seven items asking about 
students’ perceptions of their influence at school. Four of these questions focused on 
general confidence in school participation (Torney-Purta et al., 2001). ICCS 2009 used 
a set of four (partly modified) CIVED items and one additional item reflecting student 
attitudes toward the value of student participation in civic-related activities at school. 
Most students across participating countries valued student participation at school, 
and females tended to be more supportive than male students (Schulz et al., 2010b). 
ICCS 2016 uses a set of five items (including four from the previous survey) to measure 
students’ attitudes toward participation in school activities. 


Students’ attitudes toward political consumerism in Europe: Over the past 20 years, 
political or ethical consumerism has emerged as an important part of citizenship 
engagement; this refers to the buying or boycotting of products or services for political 
or ethical reasons (Micheletti, & Stolle, 2004; Stolle, Hoghe, & Micheletti, 2005). 
Political consumerism is defined as the choice of producers and products with the 
intention of changing institutional or market practices (Micheletti, & Stolle, 2015). 
The ICCS 2016 European regional questionnaire includes a question asking students to 
rate their agreement or disagreement with several statements about political or ethical 
consumerism. 


Students’ attitudes toward corrupt practices in Latin America: Corruption is widely 
regarded as one of Latin America’s most salient problems and, with few exceptions, 
countries in this region tend to have low indices of transparency in cross-national surveys 
(Transparency International, 2014). Citizens’ perceptions of the level of corruption 
have also been found to be related to lower levels of trust in institutions (Morris, & 
Klesner, 2010). Furthermore, large proportions of Latin American citizens reported 
in regional surveys direct experiences with corrupt practices (Morris, & Blake, 2010) 
and the World Values Survey found that in this region acceptance of corruption was 
relatively higher than in other countries (Torgler, & Valev, 2004). In its Latin American 
regional questionnaire, ICCS 2009 gathered data about young people’s attitudes toward 
corrupt practices, and results showed an acceptance of corrupt practices by many, albeit 
not a majority of students (Schulz, Ainley, Friedman, & Lietz, 2011). The ICCS 2016 
Latin American regional questionnaire includes the same question as in the previous 
survey, and will allow changes in attitudes since 2009 to be monitored. 


Students’ attitudes toward violence in Latin America: Among the pressing problems that 
Latin American societies are facing, violence and crime are those with wide-reaching 
consequences for young people’s socialization into society (Reimers, 2007). Exposure 
to violence has been identified as causing higher levels of aggressive and violent 
behavior among young people (Chaux, 2009; Chaux, & Velazquez, 2009). Young people 
with supportive attitudes towards violence are more likely to participate in violent 
behavior themselves (Copeland-Linder, Johnson, Haynie, Chung, & Cheng, 2012). 
The ICCS 2009 Latin American questionnaire asked students to rate their agreement 
or disagreement with a series of statements about the use of violence. While most 
students rejected positive statements about the use of violence, males tended to be more 
supportive of violence than females (Schulz, Ainley, Friedman, & Lietz, 2011). In the 
Latin American regional questionnaire for ICCS 2016, students are presented with the 
question augmented by two further new items designed to measure students’ attitudes 
toward the use of violence. 
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Students’ attitudes toward civic identities 


The following constructs reflecting student attitudes toward civic identities were 
measured as part of the international student questionnaire or the European and Latin 
American regional questionnaires in ICCS 2009: 


+ Students’ attitudes toward their country of residence 
* Students’ sense of European identity (European regional questionnaire) 


+ Students’ perceptions of their own individual future (European regional 
questionnaire) 


+ Students’ acceptance of diversity (Latin American regional questionnaire). 


Students’ attitudes toward their country of residence: This construct reflects students’ 
attitudes toward abstract concepts of nation. Various forms of national attachment may 
be distinguished (symbolic, constructive, uncritical patriotism, or nationalism), which 
differ from feelings of national identity (Huddy, & Khatib, 2007). Kennedy (2010) 
argued that students in Hong Kong viewed citizenship as involving legal obligations to 
authorities, personal obligations to support others, and patriotic obligations to support 
the nation state. The CIVED survey included 12 items reflecting attitudes toward the 
students’ country. Four of these items were used to measure a construct called positive 
attitudes toward one’s nation (Torney-Purta et al., 2001) while another set of four items 
reflected protective nationalism (Barber, Fennelly, & Torney-Purta, 2013). ICCS 2009 
useda set of eight items (four of them from CIVED) to measure students attitudes toward 
the country they live in. The results showed that large majorities across participating 
countries endorsed positive statements about their countries of residence; however, 
notable differences were recorded between young people with and without immigrant 
backgrounds (Schulz et al., 2010b, pp. 101-104). ICCS 2016 assesses attitudes toward 
their country (of residence) using a slightly reduced set of items measuring students’ 
attitudes toward their country of residence. 


Students’ sense of European identity: European identity and its citizens’ sense of belonging 
have been important themes of debate over the past decade within the EU (Alnæs, 
2013; Checkel, & Katzenstein, 2009; European Commission, 2012b; Delanty 1995, 
2006; Delanty, & Rumford, 2005; Duchesne, 2008; Herrmann, Risse, & Brewer, 2004; 
Karolewski, & Kaina, 2006, 2013; Pichler, 2008; Spannring, Wallace, & Datler, 2008). 
The establishment of European institutions and integration of EU member countries, 
and in particular the signing of the Treaty on the European Union (better known as 
the Treaty of Maastricht), have had consequences for the concept of European identity 
and citizenship (Osler, & Starkey, 2008). While some scholars claim that supra-national 
identities have superseded national identities (see for example, Osler, & Starkey, 
2001, 2008; Soysal, 1994), others hold that notions of national citizenship still remain 
dominant (Delanty, 2007; Fligstein, 2009). The European regional questionnaire of 
ICCS 2009 included a question about the extent to which lower-secondary students 
have developed a sense of European identity. Results showed that, while most students 
regarded themselves as Europeans, relatively few students viewed their European 
identity as more important than their national identity (Kerr et al., 2010). The European 
regional questionnaire for ICCS 2016 includes the same question as in the previous 
survey in order to measure changes in the sense of European identity over time. 
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Students’ perceptions of their own individual future in European societies: In a previous 
part of the framework, we drew attention to the need to examine perceptions of the 
future held by students. There is a body of literature concerned with the measurement 
of beliefs about, and perceptions of the future or future time perspectives (Husman, 
& Shell, 2008). This measurement goes beyond simple measures of dispositional 
optimism and pessimism (Lemola, Raikkonen, Mathews, Schier, Heinonen, Pesonen, 
& Lahti, 2010). Examining perceptions of the future involve an element of appraisal, as 
well as a response to that appraisal. We have already noted the evidence that adults in 
European countries think that life for the next generation will be more difficult that it 
was for themselves (European Commission, 2014). The ICCS 2016 European regional 
questionnaire asks students about the likelihood of finding employment and better 
financial conditions in the future. 


Students’ acceptance of diversity in Latin America: Acceptance of minority groups and the 
rejection of discrimination can be viewed as essential for ensuring the well-being for all 
members of society, as well as an educational goal (Cabrera, Nora, Terenzini, Pascarella, 
& Hagedorn, 1999; Cote, & Erikson, 2009; Morley, 2003). An example of the integration 
of this educational aim for civic and citizenship education is the Colombian program 
Citizenship Competencies, which encompasses learning about pluralism, identity, and 
respect for diversity, as well as issues related to exclusion and discrimination (Chaux, 
Lleras, & Velazquez, 2004; Ministry of Education of Colombia, 2004). The ICCS 
2009 Latin American questionnaire included a set of items measuring the acceptance 
of different social minority groups as neighbors. While most students across the six 
participating countries were tolerant of people with different nationality, from other 
regions of the country, or with a different skin color, fewer students approved of people 
with a different sexual orientation in their neighborhood (Schulz, Ainley, Friedman, 
& Lietz, 2011). Acceptance of social minority groups tended be positively associated 
with civic knowledge, and had negative correlations with authoritarian attitudes (Caro, 
& Schulz, 2012). The Latin American regional questionnaire for ICCS 2016 includes 
a modified set of items measuring students’ acceptance of social minorities in their 
neighborhood. 


2.5.2 Affective-behavioral domain 2: Engagement 


The affective-behavioral domain Engagement refers to students’ civic engagement, 
students’ expectations of future action, and their dispositions to actively engage in 
society (interest, sense of efficacy). This affective-behavioral domain, assessed in 
the student perceptions questionnaire, requires items that ask students about their 
intentions toward civic action in the near future or when they are adults, as well as items 
measuring the extent to which students are interested and feel competent to engage. 
Given the age group to be surveyed in ICCS 2016 and the limitations that adolescents 
face in participating as active citizens, students’ dispositions towards engagement are 
of particular importance when collecting data about active citizenship. In addition to 
active involvement in those civic forms open to this age group (such as school-based 
activities, youth organizations, or community groups), young people may now become 
involved in virtual networks through new social media. These relatively new forms of 
engagement are considered more explicitly in ICCS 2016. 
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While indicators of engagement are mainly related to the content domain civic 
participation, they are also concerned with other content domains (mainly at the 
level of individual items). For example, a student’s expected membership in a political 
party is related to the content domain civic society and systems, a student’s expected 
engagement in political consumerism to the content domain civic principles, and a 
students’ participation in a group to help the local community to civic identity. 


One important aspect of measuring dispositions toward civic engagement in the area 
of civics and citizenship, which has traditionally been a central focus in political science 
research, is political participation. It can be defined as “activity that has the intent or 
effect of influencing government action—either directly by affecting the making of 
implementation of public policy or indirectly by influencing the selection of people 
who make those policies” (Verba et al., 1995, p. 38). Putnam (1993, p. 665) regarded civic 
engagement more broadly as “people’s connections with the life of their communities, 
not merely politics.” 


Verba et al. (1995) identified the following three factors as predictors of political 
participation: (i) Resources enabling individuals to participate (time, knowledge); (ii) 
psychological engagement (interest, efficacy); and (iii) “recruitment networks,” which 
help to bring individuals into politics (these networks include social movements, 
church, groups, and political parties). Inequality in citizens' opportunities for political 
participation has been raised as an issue for democracy, in particular in the United 
States (see for example Schlozman, Verba, & Brady, 2012). There is a general consensus 
regarding the importance of formal education in influencing the extent of adult 
engagement in society (see Nie, Junn, & Stehlik-Barry, 1996; Pancer, 2015). 


During the 1970s and 1980s, protest behavior as a form of participation became more 
prominent in Western democracies (Barnes, & Kaase, 1979). Scholars have distinguished 
“conventional” (voting, running for office) from “unconventional (social movement)” 
activities (grass-root campaigns, protest activities). They have also distinguished 
among the latter legal from illegal forms of behavior (Kaase, 1990). In view of the rapid 
expansion of new types of political activities, Van Deth (2014) proposed a classification 
of political participation, which, in addition to conventional and unconventional types 
of engagement, also included problem- or community-oriented forms of participation 
and individualized and creative modes of participation. 


According to Ekman and Amna (2012), it is necessary to distinguish civic participation 
(latent political participation) from manifest political participation, as well as individual 
forms from collective forms of engagement. Ekman and Amna distinguished forms 
of latent involvement (such as interest and attentiveness) from more active forms of 
engagement (defined as either individual or collective activities). With regard to political 
passivity, which has been observed as a growing phenomenon especially among young 
people, it is important to distinguish unengaged from disillusioned citizens (Amna, & 
Ekman, 2014). While unengaged passive citizens are still keeping themselves informed 
and are willing to consider civic engagement if needed, disillusioned passive citizens 
have lost faith in the possibility of influencing and have become alienated. Therefore, 
in addition to active engagement, basic dispositions toward engagement (interest or 
self-efficacy) and behavioral intentions (underlying preparedness to take action) are of 
crucial importance when studying young people's engagement. 
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In recognition of the above, and also in view of the fact that young people at age of 13- 
15 years are limited with respect to the extent in which they can participate in society, 
indicators of engagement are conceptualized according to the following typology: 


e Dispositions 
* Behavioral intentions 


e Civic participation. 


Dispositions 
With regard to students’ dispositions toward civic engagement, ICCS 2016 will 
distinguish the following dispositions toward engagement: 


* Students’ interest in political and social issues 


+ Students’ sense of citizenship self-efficacy. 


Students’ interest in political and social issues: The first IEA Civic Education Study in 
1971 included measures of interest in public affairs television, which turned out to be 
a positive predictor of civic knowledge and participation (Torney et al., 1975). An item 
on political interest was used in the CIVED survey (Torney-Purta et al., 2001). Similar 
to earlier findings, CIVED results also showed interest in politics as a positive predictor 
of civic knowledge and likelihood to vote (Amadeo et al., 2002). ICCS 2009 used a list of 
items covering students’ interest in a broader range of six different political and social 
issues, including an optional item referring to interest in European politics. The results 
showed that students tended to have considerable interest in social and also political 
issues in their own countries, but were less interested in international politics (Schulz et 
al., 2010b). ICCS 2016 will assess students’ interest using an additional item about their 
interest in political and social issues, in conjunction with a question about their parents’ 
interest in these issues. 

Students’ sense of citizenship self-efficacy: This construct reflects students’ self-confidence 
in active citizenship behavior. Individuals’ “judgments of their capabilities to organize 
and execute courses of action required to attain designated types of performances” 
(Bandura, 1986, p. 391) are deemed to have a strong influence on individual choices, 
efforts, perseverance, and emotions related to the tasks. The concept of self-efficacy 
constitutes an important element of Bandura’s social cognitive theory about the learning 
process, in which learners direct their own learning (Bandura, 1993). The distinction 
between self-concept regarding political participation (political internal efficacy) and 
citizenship self-efficacy is that: whereas internal political efficacy asks about global 
statements regarding students’ general capacity to act politically, citizenship self- 
efficacy asks about the students’ self-confidence to undertake specific tasks in the area 
of civic participation. ICCS 2009 included seven items reflecting different activities that 
were relevant for students of this age group, which are also included in the ICCS 2016 
student questionnaire. 


Behavioral intentions 
ICCS 2016 will distinguish between the following three types of behavioral intentions: 


+ Expectations to participate in legal and illegal forms of civic action in support of or 
protest against important issues 


+ Expectations of political participation as adults 


° Expectations of participating in future school-based activities. 
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Students' expectations to participate in forms of civic action: In ICCS 2009 a set of nine 
items reflected students’ expectations for future involvement in protest activities (for 
example, collecting petitions, participating in protest marches, blocking traffic). The 
items related to two different dimensions of protest behavior: legal and illegal. In ICCS 
2016 items measure forms of civic action, including those in protest against and in 
support of particular issues, using a similar set of items. The items include actions in 
support of environmental sustainability, as well as use of new social media. 


Students' expectations of political participation as adults: Young people who intend to 
participate in political activities have been shown to be much more likely to actually 
participate at a later point in time (Eckstein, Noack, & Gniewosz, 2013). In ICCS 2009 
these types of behavioral intentions were measured with set of nine items (two of 
which were optional for countries), which was used to measure two different constructs 
(expected electoral participation and expected participation in political activities). 
While majorities of students across participating countries expected to participate in 
elections, relatively few students expressed intentions to engage in more active forms 
of political participation (Schulz et al., 2010b, pp. 143-146). The ICCS 2016 student 
questionnaire includes the same set of ICCS 2009 items, augmented by a number of 
items measuring more informal ways of citizen participation in society (including one 
new item regarding personal efforts to help the environment). 


Students' expectations of participating in future school-based activities: The theory of 
planned behavior links attitudes to behaviors through intentions (Ajzen, 2001; Ajzen, 
& Fishbein, 2000). This theory posits that attitudes influence actions through reasoned 
processes (that are manifested as intentions). For example, intentions formed relatively 
early in secondary school are powerful predictors of subsequent participation in 
education (Khoo, & Ainley, 2005). More specifically related to citizenship, Keating and 
Janmaat (2015) reported analyses of longitudinal data in the United Kingdom, and 
suggested that participation in school-based political activities has a positive influence 
on future electoral and political engagement. A set of items measuring this construct 
were developed for ICCS 2016 to reflect students’ beliefs about their expectation of 
undertaking future civic activities within the school context (for example, voting in 
school elections or engaging in a public debate about school-related issues). 


Civic participation 

Students at the age group under study in ICCS are not yet old enough to have access to 
many forms of citizenship participation in society. However, there is evidence of links 
between youth participation and later engagement as adult citizens (see for example, 
Verba et al., 1995). Furthermore, having been part of civic-related activities at school 
has been suggested as factor influencing future citizenship engagement (Pancer 2015; 
Putnam, 2000). In view of the latter, it needs to be acknowledged that current or past 
involvement in youth groups, school governance or campaigns may play a role as 
contextual factor in determining civic-related learning outcomes. 


ICCS 2016 includes measures of the following types of more active students’ civic 
engagement: 

* Students’ engagement with social media 

+ Students’ engagement in organizations and groups (outside of school) 


+ Students’ engagement in school activities. 
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Students’ civic participation through social media: The importance of social media has 
risen exponentially over the past years (Banaji, & Buckingham, 2013; Mihailidis, 2011; 
Rainie, Smith, Schlozman, Brady, & Verba, 2012; Segerberg, & Bennett, 2011) and 
research suggests a potential enhancement of civic participation among people when 
content is interactive (for example, via chat rooms or message boards) instead of the 
one-way communication of more traditional media (Bachen et al., 2008; Kahne et al., 
2011). The ICCS 2016 student questionnaire includes three new items that measure the 
extent to which students engage with political and social issues via social media. 


Students’ civic participation in the organizations and groups: Citizens’ involvement in 
organizations and groups can be seen as a clear indicator of civic engagement (Putnam, 
2000; Van Deth, Maraffi, Newton, & Whiteley, 1999). However, it can also be regarded as 
a resource for future engagement (Putnam, 1993). The ICCS 2009 student questionnaire 
asked students about their current or past participation in organizations in their 
communities, such as human-rights groups, religious associations, and/or youth clubs. 
Similar to the findings of the CIVED study in 1999 (Amadeo et al., 2002; Torney-Purta 
et al., 2001), ICCS 2009 results showed that only minorities among students reported to 
have participated in these organizations or groups (Schulz et al., 2010b, pp. 129-134). 
ICCS 2016 assesses students’ participation in the community with a slightly modified 
set of 10 items (including three optional items). 


Students’ civic participation in school activities: Numerous scholars have underlined the 
importance of students’ experience at school for developing a sense of having power to 
influence matters in the community (Bandura, 1997). Research has provided evidence 
that more democratic forms of school governance can contribute to higher levels of 
political engagement (see for example Mosher, Kenny, & Garrod, 1994; Pasek, Feldman, 
Romer, & Jamieson, 2008). The ICCS 2009 student questionnaire asked students about 
a wide range of civic-related participation at school (for example, in school councils/ 
parliaments, or in student debates). The results showed that majorities of students 
reported to have participated in many of these activities in school, and that there were 
positive associations with civic knowledge and engagement (Schulz et al., 2010b). ICCS 
2016 assesses students’ participation at school with a slightly modified set of eight items 
(including one optional item). 


2.6 Mapping items to domains 


The content domains relate to both cognitive and affective-behavioral domains. Any 
item that measures one of the two cognitive domains can be mapped to any of the four 
content domains. The same is true for items measuring any of the affective-behavioral 
constructs. Table 2.1 shows how items can be placed in different cells and mapped to 
either cognitive or affective-behavioral domains, as well as to content domains. 


Cognitive items from both domains (knowing and reasoning and applying) and affective- 
behavioral items from two domains (attitudes and engagement) were developed in 
the contexts of all four content domains. Because these mappings are guided by the 
compatibility of each content domain to the different affective-behavioral and cognitive 
domains, they do not necessarily spread evenly across the content domains. 
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Table 2.1: Relationship between cognitive or affective-behavioral and content domains 


Content Content Content Content 
domain 1: domain 2: domain 3: domain 4: 
Civic society Civic principles Civic participation Civic identities 
and systems 
Cognitive 
domains 
Knowing ll Ill IV 
Reasoning and 
applying V VI VII VIII 
Affective- 
behavioral 
domains 
Attitudes A B C D 
Engagement E Fe G H 


The following examples illustrate the mapping of items to domains (see Chapter 4 for 
information on the coverage of framework domains in the ICCS 2016 main survey 
instruments): 


+ A cognitive item that measures student knowledge about the role of parliament 
would be located in cell I (Cognitive domain: Knowing; Content domain 1: Civic 
society and systems). 


°- A cognitive item measuring student ability to identify the underlying reason for a 
civic protest would be found in cell VII (Cognitive domain: Reasoning and applying; 
Content domain 3: Civic participation). 


* An affective-behavioral item asking about students’ valuing of their country’s flag 
would be located in cell D (attitude related to Content domain 4: Civic identities). 


+ An affective-behavioral item asking about students’ trust in parliament would be 
located in cell A (attitude related to Content domain 1: Civic society and systems). 


+ An affective-behavioral item asking about students’ expectations to vote in national 
elections would be located in cell G (engagement item related to Content domain 3: 
Civic participation). 

+ An affective-behavioral item asking about students’ interest in political and social 
issues would be located in cell E (engagement item related to Content domain 1: 
Civic society and systems). 
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3 Contextual Framework 


3.1 Contexts for civic and citizenship education 


A study of civic-related learning outcomes and civic engagement needs to take the 
context in which civic and citizenship education occurs into account. Young people 
develop their understandings about their roles as citizens in contemporary societies 
through activities and experiences that take place within homes, schools, classrooms, 
and the wider community. It is therefore important to recognize that young people’s 
cognitive and affective-behavioral learning outcomes are potentially influenced by 
variables that can be located at different levels in a multi-level structure (see Travers, 
Garden, & Rosier, 1989; Travers, & Westbury, 1989; Scheerens, 1990; Scheerens, & 
Bosker, 1997). 


The individual student is located within overlapping contexts of school and home. Both 
contexts form part of the local community that, in turn, is embedded in the wider 
sub-national, national, and international contexts. The contextual framework for ICCS 
distinguishes the following levels: 


* Context of the wider community: This level comprises the wider context within which 
schools and home environments work. Factors can be found at local, regional, and 
national levels. For some countries, the supra-national level might also be relevant 
as, for example, in member countries of the European Union. Given the increased 
importance of new social media, virtual communities connected through the internet 
also form part of this context. 


* Context of schools and classrooms: This level comprises factors related to the instruction 
students receive, the school culture, and the general school environment.” 


* Context of home and peer environments: This level comprises factors related to the 
home background and the immediate social out-of-school environment of the 
student (for example, peer-group activities). 


* Context of the individual: This level refers to the individual characteristics of the 
student. 


Another important distinction can be made by grouping contextual variables into 
antecedents or processes: 


+ Antecedents are those variables that shape how student learning and acquisition of 
civic-related understandings and perceptions takes place. Note that these factors are 
level-specific and may be influenced by antecedents or processes at a higher level. For 
example, civic-related training of teachers may be affected by historical factors and/ 
or policies implemented at the national level. 


* Processes are those variables related to civic-related learning and the acquisition of 
understandings, competencies, and dispositions. They are constrained by antecedents 
and possibly influenced by variables relating to the higher levels of the multi-level 
structure. 


10 Because of the sampling design for ICCS, school level and classroom level cannot be disentangled. Generally, only one 
classroom will be selected within each sampled school. 
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Antecedents and processes are variables that have potential impact on the outcomes at 
the level of the individual student. Learning outcomes related to civic and citizenship 
education at the student level also can be viewed as aggregates at higher levels (school, 
country) where they can affect factors related to process. For example, higher levels of 
civic understanding and engagement among students may influence the way schools 
teach civic and citizenship education. 


Figure 3.1 illustrates contextual variables which might influence the learning outcomes 
of civic and citizenship education. There is a reciprocal relationship between processes 
and outcomes, which emphasizes that “feedback” may occur between civic-related 
learning outcomes and processes. For example, students with higher levels of civic 
knowledge and engagement are those students most likely to participate in activities (at 
school, at home, and within the community) that promote these outcomes. 


There is a unidirectional relationship between antecedents and processes at each level. 
However, higher-level processes may influence antecedents, and it is likely that, from a 
long-term perspective, outcomes may affect variables that are antecedents for learning 
processes. 


Figure 3.1: Contexts for the development of learning outcomes related to civic and 
citizenship education 


Antecedents Processes Outcome 
Wider community Wider community 
Education system Educational policies 
History and culture Political events 
School/classroom: School/classroom: 
Characteristics Instruction 
Composition Governance 
Resources 
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CONTEXTUAL FRAMEWORK 


This contextual framework for ICCS makes it possible to map variables for which data 
are collected on a three-by-four grid, with antecedents, processes, and outcomes as 
columns, and the levels of country/community, school/classroom, student, and home 
environment as rows (Table 3.1). Although the last column for outcomes is not split 
into levels, it is important to recognize that, for the analysis, aggregates can also be used 
at wider community or school/classroom levels." 


Table 3.1 shows examples of potential variables (or groups of variables) collected with 
different ICCS instruments for each cell in this grid. Variables related to the context of 
country/community are collected primarily through the national contexts survey and 
other possible data sources. Variables related to the context of schools and classrooms 
are collected through the school and teacher questionnaires. The student background 
questionnaire provides information on antecedents of the individual student and the 
home environmentas well as about some process-related variables (for example, learning 
activities). The student test and the student perceptions questionnaire will collect data 
on outcomes. In addition, the student background questionnaire will include questions 
regarding student participation in civic-related activities, which will also be used as 
indicators of active citizenship related to Content domain 3 (civic practices). 


Some potential variables that can be measured at one level pertaining to another 
level are not included in the mapping shown in Table 3.1. Student observations of 
learning practices in the classroom can be aggregated and used as classroom or school 
variables. Student, school, and teacher questionnaires might also provide civic-related 
information about the context of the local community. 


Table 3.1: Mapping of variables to contextual framework (examples) 


Level of ... Antecedents Processes Outcomes 
Wider NCS & other sources: NCS & other sources: 
community Democratic history Intended curriculum 
Structure of education Political developments 
School/classroom ScQ & TQ: ScQ & TQ: 
School characteristics Implemented 
Resources curriculum StT & StQ/RQ: 


Policies and practices Test results 


Student attitudes 

Student Sta: StQ: and engagement 

Gender Civic learning 

Age Practiced engagement 
Home and peer Sta: StQ: 
environment Parent SES Family communication 

Ethnicity Communication 

Language with peers 

Country of birth Media information 


Note: NCS = national contexts survey; ScQ = school questionnaire; TQ = teacher questionnaire; RQ = regional 
questionnaire; StQ = student questionnaire; StT = student test; SES = socioeconomic status. 


11 Note that similar conceptualizations have been used for the planning of other international studies (see, for example, 
Harvey-Beavis, 2002; OECD, 2005; Travers, &Westbury, 1989; Travers et al., 1989). 
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3.2 The context of the wider community 


The context of the wider community can be viewed as consisting of different levels: the 
local community in which students’ schools, as well as home and peer environments, 
are embedded within broader contexts of regional, national, and possibly supranational 
contexts. Within the scope of ICCS, the level of the community and the level of the 
national context are the most relevant levels. 


3.2.1 The context of the educational system 


To study the ways students develop civic-related dispositions and competencies and 
acquire understandings with regard to their role as citizens, it is important to take 
variables found at the country level into account. Historical background, the political 
system, the structure of education, and the curriculum provide important contextual 
information when interpreting results from an international assessment of civic and 
citizenship education. Data from official statistics will provide a range context data 
at the level of countries regarding the structure of the education system, the nature 
of the political system, and the economic and social context of the society. However, 
comparable data from published sources will not always be available to provide a 
picture of the context for civic and citizenship education in all participating countries. 


The national contexts survey for ICCS 2016 is designed to collect systematically relevant 
data that are not always available from existing sources. These data include information 
on the structure of national education systems, education policies and approaches to 
civic and citizenship education, teacher training in general and for civic and citizenship 
education in particular, and approaches to assessment and quality assurance regarding 
the area of civic and citizenship education. The survey also collects information on 
current debates and reforms regarding this learning area. 


Data from published sources and from the national contexts survey will be used to 
compare profiles of civic and citizenship education in participating countries. In 
addition, national context data will be used for the analysis of differences among 
countries in student knowledge and engagement related to civic and citizenship 
education. 


The structure of the education system: Despite a number of global trends in education 
that have led to similarities in policies and structures (Benavot, Cha, Kamens, Meyer, 
& Wong, 1991), the differences between education systems continue to have a 
considerable effect on the outcomes of education (Baker, & LeTendre, 2005). To capture 
these basic differences, ICCS 2016 collects data on the structure of school education 
(study programs, public/private school management, types of primary and secondary 
education institutions), the autonomy of educational providers, and the length of 
compulsory schooling. 


Education policies regarding civic and citizenship education: Results from ICCS 2009 
(Ainley et al., 2013) showed that the status of and priority given to civic and citizenship 
education were mostly regarded as low across participating countries. Generally, civic 
goals were deemed as important, however, there were varying approaches regarding 
the delivery of curricular content across countries, either through its integration into 
different subjects, teaching as part of a distinct subject, and/or definition as a cross- 
curricular learning area. ICCS 2009 findings also highlighted the fact that explicit civic 
and citizenship education often starts after students reached the age of 14. 


CONTEXTUAL FRAMEWORK 


The last decade has witnessed numerous examples of educational reforms in many 
countries, with the overall aim of improving educational provision and outcomes, 
including those concerning civic and citizenship education. Many of these educational 
reforms were implemented in response to the challenges of learning and living in 
modern societies, as well as changes in political systems (Ainley et al., 2013; Cox, 
Jaramillo, & Reimers, 2005). 


The ICCS 2016 national contexts survey collects data on the definition of, and the 
priority given to, civic and citizenship education in the educational policy and its 
provision in each participating country at the time of the data collection. National 
centers will provide information about official definition of civic and citizenship 
education, its place in the curriculum in primary and secondary education, and its 
main goals. National centers are also asked about the potential influence of historical, 
cultural, political, and other contexts on the character of and approach to civic and 
citizenship education, and whether there have been any changes since the previous 
survey in 2009. 


Civic and citizenship education and school curriculum approaches: Countries take 
different approaches to the implementation of civic and citizenship education in their 
curricula and the ways civic and citizenship education is generally implemented vary 
considerably across countries (Ainley et al., 2013; Cox et al., 2005; Eurydice, 2005). Some 
educational systems have it in the national curriculum as a compulsory or optional 
(stand-alone) subject, whereas others include it through integration into other subjects. 
An alternative approach to civic and citizenship education is to implement it as a cross- 
curricular theme or through the so called “whole school approach”. ICCS 2009 results 
showed that in many education systems and/or schools more than one approach is 
implemented at the same time (Ainley et al., 2013). 


With regard to school curriculum approaches for civic and citizenship education, 
Eurydice (2012) distinguished (i) promotion through steering documents such a 
national curricula or other recommendations/regulations, (ii) support for school-based 
programs and projects, and (iii) the establishment of political structures (such as school 
parliaments). In this context it is also important to review the extent to which schools in 
different countries provide support for civic and citizenship education through school 
culture or ethos, democratic school governance, and the establishment of links with the 
wider community (Birzea et al., 2004; Eurydice, 2012). Results from ICCS 2009 showed 
that many countries include recommendations with regard to the establishment of 
democratic school practices in their educational policies (see Ainley et al., 2013). 


The national contexts survey in ICCS 2016 gathers data on the inclusion of civic and 
citizenship education (as a separate subject, or integrated into different subjects, or as 
cross-curricular approach) in the formal curriculum at different stages of schooling and 
in different study programs. The survey also captures the names of specific curriculum 
subjects and whether they are compulsory or optional in each study program. 
Furthermore, the national contexts survey gathers data on goals of the national or 
official curricula for civic and citizenship education regarding the inclusion of specific 
contexts with regard to whole school approaches, school curriculum approaches, 
student participation or parental involvement, and links to the wider community. 
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Because ICCS 2016 surveys students at a specific target grade in lower secondary 
programs (typically Grade 8), it will be important to gather information about 
the curricular context for civic and citizenship education in this particular grade. 
Furthermore, national centers are asked to report on the specification of topics, objectives 
and processes when implementing the school curriculum, as well as specifications 
regarding the amount of instructional time given to civic and citizenship education. 


Teachers and civic and citizenship education: The teacher survey undertaken as part of 
the CIVED survey showed a great deal of diversity in the subject-matter background, 
professional development, and work experience of those teachers involved in civic and 
citizenship education (Losito, & Mintrop, 2001). With regard to teacher training in this 
field, research showed a rather limited and inconsistent approach to in-service training 
and professional development (Birzea et al., 2004; Eurydice, 2005, 2012). The results of 
the ICCS 2009 national contexts survey showed that, in most participating countries, 
pre-service and in-service training was provided but, in most cases, this provision was 
reported as non-mandatory (Schulz et al., 2010b, pp. 53—56). 


To assess the variety of different approaches to teacher education in the field at the 
level of education systems, the national contexts survey in ICCS 2016 collects general 
data about the requirements for becoming a teacher and about licensing or certification 
procedures for teachers. More specifically, the survey also gathers data about the 
characteristics of teachers of civic and citizenship education and the extent to which 
civic and citizenship education is part of pre-service or initial teacher education, and 
on the availability of in-service or continuing professional development education in 
general, and for civic and citizenship education in particular, from the providers of 
these activities. 


Assessment and quality assurance in civic and citizenship education: Comparisons of 
assessment and quality assurance for civic and citizenship education are difficult and 
complex due to the diversity of approaches to teaching this subject area across countries. 
In particular, research in Europe shows that, in most countries, and compared to other 
subject areas, monitoring and quality assurance in civic and citizenship education are 
often unconnected and carried out on a small scale (Birzea et al., 2004). However, over 
the last decade, some countries have started to implement nationwide assessments of 
civic and citizenship education (Ainley et al., 2013; Eurydice, 2012). 


The national contexts survey includes questions about the extent of assessment in the 
area of civic and citizenship education at the country’s target grade, and how parents 
are informed about current approaches to this field of learning. 


3.2.2 The context of the local community and school-community 
relationships 


Schools and homes of students are located in communities that vary in their 
economic, cultural, and social resources, and in their organizational features. Inclusive 
communities that value community relations and facilitate active citizen engagement, 
especially if they are well resourced, may offer civic and citizenship opportunities for 
partnerships and involvement to schools and individuals. Social and cultural stimuli 
arising from the local community, as well as the availability of cultural and social 
resources, may influence young people’s civic and citizenship knowledge, dispositions, 
and competencies in relation to their roles as citizens (Jennings, Stoker, & Bowers, 
2001). Data on the contexts and characteristics of the local community will be gathered 
primarily through the school questionnaire. 


CONTEXTUAL FRAMEWORK 


Urbanization: There is evidence that students from non-urban school contexts often 
perform at lower achievement levels than those from urban schools (see, for example, 
Istrate, Noveanu, & Smith, 2006; Webster, & Fisher, 2000; Williams, 2005). Data on 
school location (urbanization) were used in multi-level analyses carried out in ICCS 
2009. In most countries, a rural school location had no significant effect on students’ 
civic knowledge, after controlling for other variables (see Schulz et al., 2010b, pp. 230- 
232). Urbanization was associated with student knowledge in only a few countries. 
In Latin American countries, there were significant differences in civic knowledge 
between rural and urban schools that were largely associated with the socioeconomic 
background of individual students and their schools (Schulz, Ainley, Friedman, & Lietz, 
2011, p. 78). As in ICCS 2009, the ICCS 2016 school questionnaire includes a question 
about the size of the community in which the school was located. 


Availability of resources in the local community: Differences in the quantity and quality 
of resources for citizenship learning available in the local area may have a dual effect. 
On the one hand, they may favor the organization of community-oriented projects 
and student participation in projects requiring the development of activities involving 
the community, both of which can contribute to developing skills and competencies 
related to civic and citizenship education. On the other hand, community participation 
in the life of the school and in its various levels can be a factor for greater openness and 
democratization of the school itself. Furthermore, the level of resources may influence 
the possibilities for the provision of local support to schools, which may have an 
impact on school improvement (Reezigt, & Creemers, 2005). In ICCS 2009, differences 
regarding the availability of resources in the local community were associated with 
students’ civic knowledge in several countries (see Schulz et al., 2010b). They also 
provided an additional measure of the schools’ economic and social contexts. The 
question used in ICCS 2009 is also included in the ICCS 2016 school questionnaire, 
with minor modifications. 


Issues of social tension in the community: As part of the community within which it is 
located, the school may be affected by issues and problems existing at the community 
level. Issues of social tension within the local community may influence students’ 
social relationships and the quality of their social lives and everyday experiences, both 
outside and inside the school (LHomme, & Jerez Henríquez, 2010). In addition to that, 
students’ actual opportunities to volunteer or participate in civic-related activities in the 
communities may be influenced by the social climate existing in the local communities 
within which schools are located. A safe social environment is likely to enhance students’ 
activities and participation in the local community. Conversely, issues creating social 
tensions and conflicts in the local community may discourage students’ involvement 
in civic activities. In ICCS 2009, principals were asked about their perceptions of social 
tension in the community, and the results showed a negative association between higher 
levels of perceived social tension and students' civic knowledge (Schulz et al., 2010b, pp. 
164-165). The ICCS 2016 school questionnaire includes a similar question, with minor 
modifications, to that used in the previous survey. 


Students’ participation in civic-related activities in the local community: Research has 
illustrated the importance of students’ activities in the community and their reflection 
on them for the construction and the development of knowledge and skills for active 
citizenship (Annette, 2008; Henderson, Pancer, & Brown, 2013). The links between 
the school and its community represent an opportunity for motivating student 
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participation in activities related to civic and citizenship education, and for offering 
them opportunities for civic engagement. Schools’ interactions with their local 
communities, and the links that have been established with other civic-related and 
political institutions, can also influence student perceptions of their relationship with 
the wider community and of the different roles they may play in it (Annette, 2000, 
2008; Potter, 2002; Torney-Purta, & Barber, 2004). ICCS 2009 showed that most of the 
students in almost all the participating countries had at least some opportunities to 
participate in such activities (Schulz et al., 2010b, pp. 154-155). The ICCS 2016 school 
questionnaire includes a modified form of the ICCS 2009 question about principals’ 
perceptions of the opportunities students have to participate in activities carried out by 
the school in cooperation with external groups or organizations. 


In ICCS 2009 the teacher questionnaire also included a question on student 
participation in civic-related activities in the local community, which was similar to 
the question included in the school questionnaire (Teachers' perceptions of student 
activities in the community). Results were generally consistent with those associated 
with principals’ answers (Schulz et al., 2010b, pp. 152-153). Comparisons between the 
principals’ and teachers’ reports provide a broader picture of what schools actually do 
from different perspectives and viewpoints. The ICCS 2016 teacher questionnaire uses 
a similar question to that in the previous survey, which asks teachers whether they had 
participated with their students in activities in cooperation with external groups or 
organizations. 


3.3 The contexts of schools and classrooms 


As in the previous survey, ICCS 2016 considers students’ learning outcomes in the 
field of civic and citizenship education not only as a result of teaching and learning 
processes, but also as the result of their daily experience at school. School experiences 
and their impact on learning outcomes are of particular importance in the context 
of civic and citizenship education, which is meant to develop learning outcomes that 
are not confined to the area of cognitive achievement, but also include attitudes and 
dispositions (Schulz et al., 2008)". A large number of countries place emphasis on 
non-formal aspects of civic learning through participation and engagement or social 
interaction at schools (Ainley et al., 2013; Eurydice, 2005, 2012; Schulz et al., 2010b). 


Students’ experience at school not only depends on the teaching and learning developed 
at a classroom level, but also on the possibilities they have to experience the classroom 
and the school as a “democratic learning environment” (through participation at a 
school level, school and classroom climate, as well as the quality of the relationships 
within the school, between teacher and students, and among students) (Backman, 
& Trafford, 2007; Huddleston, 2007; Trafford, 2003). The possibility of establishing 
and experiencing relationships and behaviors based on openness, mutual respect, 
and respect for diversity, as well as the possibility of giving and asserting personal 
opinions, allow students to practice a democratic lifestyle, to begin exercising their own 
autonomy, and to develop a sense of self-efficacy (see Mosher et al., 1994; Pasek et al., 
2008). Recent research has also stressed the importance of informal learning at school 
for the development of students’ active citizenship (Scheerens, 2009). 


12 According to the UN resolution “Education for Democracy” (United Nations, 2012), schools are not only seen as 
responsible for delivering human rights education and citizenship curricula, but also for “extracurricular educational 
activities aimed at the promotion and consolidation of democratic values and democratic governance and human rights, 
taking into account innovative approaches and best practices in the field, in order to facilitate citizens’ empowerment and 
participation in political life and policymaking at all levels.” 


CONTEXTUAL FRAMEWORK 


In view of the importance of school and classroom contexts for civic and citizenship 
education, ICCS 2016 makes use of the following types of questions: 


* School questionnaire questions measuring principals' perceptions of school contexts 
and characteristics 


* Teacher questionnaire questions about the background of teachers and their 
perceptions of school and classroom contexts 


e Student questionnaire questions about students’ perceptions of school and classroom 
contexts. 


3.3.1 School contexts and characteristics 


School climate generally refers to “the shared beliefs, the relations between individuals 
and groups in the organization, the physical surroundings, and the characteristics 
of individuals and groups participating in the organization” (Van Houtte, 2005, p. 
85). In a civic and citizenship education context, school climate can be referred to as 
“impressions, beliefs, and expectations held by members of the school community 
about their school as a learning environment, their associated behavior, and the symbols 
and institutions that represent the patterned expressions of the behavior” (Homana, 
Barber, & Torney-Purta, 2006, p. 3). A variety of learning situations can affect civic and 
citizenship education at schools. These include management, everyday activities within 
the school, the support for professional relationships inside the school itself, and the 
quality of links between the school and the outside community (Thapa, Cohen, Guffey, 
& Higgins-D’ Alessandro, 2013). 


School climate also relates to the school culture and ethos that contribute to defining 
the school as a social organization, as well as distinguishing each individual school 
from others (Stoll, 1999). School culture refers to patterns of meaning that include 
norms, beliefs, and traditions shared by the members of the school community, and 
that contribute to shaping their thinking and the way they act (Stolp, 1994). 


School climate and culture may contribute to the development among students and 
teachers of a sense of belonging to the school, thereby enhancing the commitment 
and motivation that these groups have toward improving school educational activities 
(Knowles, & McCafferty-Wright, 2015). Participative governance practices contribute 
to characterizing the schools as democratic learning environments, and promoting 
teachers’ participation in school governance helps the school to understand the variety 
of student learning needs and secure teachers’ commitment to supporting school 
educational activities (Ranson, Farrell, Peim, & Smith, 2005). 


The ICCS 2016 school questionnaire includes a wide range of questions related to 
school climate, which measure principals’ perceptions of teachers’ and students’ sense 
of belonging to the school, teachers’, students’ and parents’ participation in decision- 
making processes, teachers’ participation in school governance, the extent of bullying at 
school, and principals’ reports on activities to prevent bullying. 


Principals’ perceptions of the engagement of the school community: Different styles of 
leadership and different strategies and procedures available to principals when exercising 
their role may also impact on the school climate and culture (Edmonds, 1979; Eurydice, 
2013; Ishimaru, 2013; Marzano, 2003; Scheerens, Glas, & Thomas, 2003; Sammons, Gu, 
Day, & Ko, 2011). Therefore, a study of contexts for civic and citizenship education also 
needs to investigate how principals exercise their role in relation to the development 
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of a democratic school environment, which is open to teachers’, students’, and parents’ 
participation in decision-making processes (Torrance, 2013). ICCS 2016 includes a 
question on the extent to which teachers, parents and students are involved in decision- 
making processes. The School questionnaire also includes a question on principals’ 
reports of students’ participation as class representatives and in school elections that 
had been included in ICCS 2009. 


Principals’ perceptions of teacher participation in school governance: Empowering teachers 
to participate in decision-making at schools may contribute to active citizenship 
behavior within schools (Bogler, & Somech, 2005). The ICCS 2009 school questionnaire 
included seven items concerned with principals’ perceptions of teacher participation 
in school governance. These questions were intended to provide information about 
the extent to which teachers were willing to accept responsibilities beyond teaching. 
Using a modified question, the ICCS 2016 school questionnaire measures perceptions 
of principals regarding teachers’ participation in school governance, teachers’ support 
for maintaining good discipline, and teachers’ willingness to become members of the 
school council. 


Principals’ perceptions of bullying at school: Bullying is defined as including aggressive 
behaviors intended to hurt someone either physically, emotionally, verbally or through 
the internet (AERA [American Educational Research Association], 2013; Olweus, 1973; 
Wade, & Beran, 2011). In the ICCS 2016 school questionnaire, principals are asked to 
report on the frequency of aggressive behaviors they observe within the school. 


Principals’ reports on activities to prevent bullying: Schools are currently facing the 
problem of bullying both in the school context and in a cyber context (AERA, 2013; 
Corcoran, & Mc Guckin, 2014). Research has shown that bullying shows considerable 
variation between classes within schools (Atria, Strohmeier, & Spiel, 2007; Salmivalli, 
2012). Although a “culture of silence” still persists among victims, activities to highlight 
bullying seem to have an impact, and may help to reduce bullying inside schools 
(Smith, & Shu, 2000); prevention programs seem to have greater effect at the classroom 
level than at the school level (Karna et al., 2011). The school questionnaire includes 
a question on the initiatives implemented by schools intended to prevent bullying, 
including specific professional training aimed at the prevention of “cyberbullying” 
(Wade, & Beran, 2011). 


Principals’ reports on activities related to environmental sustainability: Education for 
sustainable development (ESD), which aims at developing the learner’s competence as 
a community member and global citizen, is increasingly viewed as an important aspect 
of citizenship education (Huckle, 2008). ESD is intended to be interdisciplinary and 
holistic, and therefore should be represented throughout the curriculum. In view of this 
aim, it is argued that it needs to involve the whole school community rather than just 
being a teacher-driven activity (Henderson, & Tilbury, 2004). The ICCS 2016 school 
questionnaire includes questions on initiatives related to environmental sustainability. 
Principals are asked about the initiatives undertaken by the schools in order to become 
environments that respect the principles of sustainable development ("sustainable 
schools”; see Henderson, & Tilbury, 2004) and to enable students to experience these 
principles directly (for example, through school initiatives to save energy, to reduce 
and separate waste, to purchase environmentally-friendly items, and, more generally, to 
encourage students’ environmental-friendly behaviors). 


CONTEXTUAL FRAMEWORK 


Principals' reports of students' access to ICT and to internet for their learning activities: 
The rapid increase in usage of internet and new social media in youth has several 
important educational implications. Formal education in new social media literacy has 
been shown to increase civic participation and provide students with access to diverse 
viewpoints (Kahne, 2010). In view of this development, ICCS 2016 also investigates 
the school context for students’ use of social media for civic engagement. The school 
questionnaire collects information about the technological resources available at school 
and about the actual access students have to them. 


Principals' reports on the delivery of civic and citizenship education at school: Many studies 
have shown that approaches to civic and citizenship education vary considerably across 
countries (Ainley et al., 2013; Birzea et al., 2004; Cox et al., 2005; Eurydice, 2005, 2012). 
Furthermore, ICCS 2009 results illustrated that different approaches to this learning 
area may actually coexist within the same schools (Schulz et al., 2010b, pp. 178-179). 
Principals from schools surveyed in ICCS 2009 provided interesting information on 
how they rated the most important aims of civic and citizenship education. Results 
showed notable differences across participating countries and that, generally, school 
principals regarded the most relevant aims of civic and citizenship education to be 
those related to the development of knowledge and skills (Schulz et al., 2010b, pp. 184- 
185). As in the previous survey cycle, the ICCS 2016 school questionnaire includes a 
set of questions on principals' reports about the way civic and citizenship education is 
delivered at their schools, on their perceptions of the importance of the aims of civic 
and citizenship education, and how specific responsibilities for civic and citizenship 
education are assigned within their schools. 


Principals’ reports on school autonomy for the delivery of civic and citizenship education: 
The school improvement literature shows that enabling some degree of autonomy 
favors the success of improvement efforts (Reezigt, & Creemers, 2005). The level of 
autonomy possessed by schools may influence the way civic and citizenship education 
is delivered at a school level (curriculum planning, choice of textbooks and teaching 
materials, assessment procedures and tools). The existence of national legislation, 
regulations and standards concerning the results that students should achieve does not 
necessarily imply that schools deliver similar programs and approaches to teaching 
(Eurydice, 2007). The time allocated to citizenship education, teacher qualifications, 
and the support the principals provide to civic and citizenship education within schools 
may vary (Keating, & Kerr, 2013; Keating, Kerr, Benton, Mundy, & Lopes, 2010). The 
ICCS 2016 school questionnaire includes a question about the schools’ autonomy to 
select textbooks, instigate student assessment procedures, plan curriculum, activities 
and projects related to civic and citizenship education, and implement teacher training. 


Principals’ report on school characteristics: School resources consist of both material and 
human resources, and there is no consensus on the extent to which these school resources 
can contribute to school development and improvement (Hanushek, 1994, 1997, 
2006). The ICCS 2009 school questionnaire included questions about the demographic 
characteristics of schools (public/private school, number of students, number of target 
grade students, and number of teachers). Research has shown associations between 
these characteristics and learning outcomes (Anderson, Ryan, & Shapiro, 1989). In the 
analysis for the ICCS 2009 Latin American report, statistically significant differences 
in civic knowledge between public and private schools were found in some countries, 
even after controlling for the socioeconomic context (Schulz, Ainley, Friedman, & Lietz, 
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2011, p. 73). As in the previous survey, the ICCS 2016 school questionnaire includes 
questions related to school characteristics like private or public school management, 
and the number of male and female students at school (overall and enrolled in the 
target grade). 


Principals’ perceptions of students' backgrounds: Research has emphasized the importance 
of the average socioeconomic family background of students at the level of individual 
schools (see, for example, Sirin, 2005). To capture the “social intake”, the ICCS 2016 
school questionnaire includes a question adopted from PIRLS 2011 (Mullis, Martin, 
Kennedy, Trong, & Sainsbury, 2009) that asked school principals to provide approximate 
percentages of students from economically disadvantaged or affluent homes. 


3.3.2 Teacher background and their perceptions of schools and 
classrooms 

The ICCS 2016 teacher questionnaire is administered to all teachers teaching at 
each country target grade regardless of their subject area. It is designed to capture 
the background of teachers, as well as a wide range of perceptions of school and 
classroom contexts. As in ICCS 2009, the ICCS 2016 teacher questionnaire includes an 
international option, with questions about civic and citizenship education at school 
and on the teaching practices actually adopted in this learning area. This part of the 
questionnaire is only completed by teachers of subjects related to civic and citizenship 
education. 


Teachers' reports on their background characteristics: Similar to ICCS 2009, the ICCS 
2016 teacher questionnaire includes a set of items asking about teachers’ demographic 
variables (gender, age) and the subject/s taught in general and at the target grade. 


Teachers’ participation in school governance: Teacher participation in school governance 
can be regarded as part of democratic governance processes at school and as a factor that 
can contribute to the characterization of the school as a democratic learning environment 
(Council of Europe, 2007). The ICCS 2009 teacher questionnaire comprised a set of 
seven items asking teachers about their participation in school governance. The items 
of the questions are the same as those included in the parallel question of the school 
questionnaire and formed a scale that was included in the international database. A 
similar question composed of five items is included in the 2016 teacher questionnaire. 
The items refer to teacher willingness to take on responsibilities besides teaching, and 
their reflections on the extent to which they are willing to cooperate with other teachers, 
cooperate to solve conflicts within the school, and engage in guidance and counseling 
activities. 


Teachers’ perceptions of bullying at school: Teacher behaviour has been identified as an 
explanatory variable of bullying at schools (Roland, & Galloway, 2002), which may 
be related to their function as role models and authorities in classroom interactions 
(Verkuyten, & Thijs 2002).The teacher questionnaire includes a question, which is (in a 
slightly modified version) also included in the school questionnaire, and is designed to 
capture teachers’ perceptions of bullying within the school (Olweus, 1973). 


Teachers’ perceptions of school climate: The school climate and the quality of the relations 
within the school (student-teacher relations and student-student relations) may 
influence student academic achievement (Bear, Yang, Pell, & Gaskins, 2014) and may 
also be associated with bullying at school (Powell, Powell, & Petrosko, 2015). The ICCS 
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2009 teacher questionnaire included two sets of items related to teachers’ perceptions 
of school climate. The items referred to teachers’ perceptions of student behaviors at 
school and to teachers’ perceptions of social problems at school. Both questions are also 
included in the 2016 teacher questionnaire. 


Teachers’ perceptions of classroom climate: Classroom climate is a general concept, where 
definitions focus mainly on the level of cooperation in teaching and learning activities, 
fairness of grading, and social support. Democratic classroom climate focuses mainly 
on the implementation of democratic and liberal values in the classroom (Ehman, 1980; 
Hahn, 1999). A democratic classroom climate may help students in understanding the 
advantages of democratic values and practices, and may have a positive effect on their 
active assimilation (Perliger, Canetti-Nisim, & Pedahzur, 2006). As some studies have 
pointed out, aside from teachers’ perceptions, what critically matters are the students’ 
perceptions of classroom climate (Hooghe, & Quintelier, 2013). The ICCS 2009 
teacher questionnaire included a set of items asking teachers about their perception 
of classroom climate and about students’ participation in classroom activities. The 
four items formed a scale that was included in the ICCS 2009 database. Results showed 
positive associations with civic knowledge in a number of countries (Schulz et al., 
2010b, p. 173). The question is also included in the ICCS 2016 teacher questionnaire. 


Teachers' perceptions of activities related to environmental sustainability: As in the ICCS 
2016 school questionnaire, the teacher questionnaire includes a question that asks 
about teachers’ involvement in initiatives and programs related to environmental 
sustainability (Kyburz-Graber, 2013; Lundholm, Hopwood, & Kelsey, 2013; UNESCO, 
2012a). The items included in the question are related to activities that enhance students 
direct involvement and engagement both within the school and in the local community, 
as well as their awareness of the impact of their behaviors on environment. 


Teachers' perceptions of the delivery of civic and citizenship education at school: The 
ICCS 2009 teacher questionnaire included two set of items related to the way civic 
and citizenship education is delivered at the school level. The two questions asked 
teachers about their perceptions of the importance of the aims of civic and citizenship 
education, and about how specific responsibilities for civic and citizenship education 
are assigned within the school. With respect to the importance of different aims of civic 
and citizenship education, results were very similar to those of the school questionnaire 
(Schulz et al., 2010b, pp. 182-183). ICCS 2016 includes slightly modified questions 
related to the aims of civic and citizenship education in both teacher and school 
questionnaires. 


Teachers' perceptions of ICT use for teaching and learning: Research has shown widespread 
use of ICT in secondary education, as well as considerable differences in the equipment 
of schools with ICT resources (see Fraillon, Ainley, Schulz, Friedman, & Gebhardt, 
2014). ICCS 2016 asks teachers to indicate whether and to what extent their schools 
provide them with a set of electronic devices with an internet connection that they can 
use for their teaching activities at the target grade. A similar question (with a focus on 
resource provisions) is also included in the school questionnaire. 


Teachers' perceptions of their teaching of subjects related to civic and citizenship education: 
Studies have shown that teacher preparation is one of the most important factors 
influencing student achievement (see OECD, 2009, 2014b). With regard to civic and 
citizenship education, teacher training is a particular challenge for educational policies, 
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and in many countries no specific training is provided to teachers in this area (Birzea 
et al., 2004; Eurydice, 2005, 2012). Following a classification developed by Shulman 
(1986, 1987), teacher knowledge may relate to topics related to civic and citizenship 
education (content knowledge), or to teaching methods and approaches (pedagogical 
knowledge). Furthermore, there are a wide range of teaching approaches in this learning 
area (Munn, Brown, & Ross, 2012). Results from ICCS 2009 showed that teachers of 
civic-related subjects tended to be most confident about teaching citizens’ rights and 
responsibilities and human rights, while they were less confident in teaching topics 
related to the economy, business and legal institutions (Schulz et al., 2010b). Also using 
questions that are identical to questions from the ICCS 2009, the section of the ICCS 
2016 teacher questionnaire administered to teachers who teach subjects related to civic 
and citizenship education collects data on the following aspects: 


* Teachers’ reports about the planning of civic and citizenship education (use of 
different sources) and about teaching and learning activities (such as “interactive 
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teaching’, “traditional” teaching, and discussion of controversial issues in classrooms). 


* Teachers’ reports on the use of different assessment tools in their teaching of civic 
and citizenship education. 


+ Teachers' preparation in teaching civic and citizenship related topics. 


* Teachers’ perceptions of possible improvements to the teaching of civic and 
citizenship education at their schools. 


* Teachers’ reports on their preparation and in-service training on topics relating 
to civic and citizenship education (content knowledge) or teaching methods and 
approaches. 


3.3.3 Student perceptions of the context of schools and classrooms 


Students’ perceptions of the school and classroom context encompass the classroom 
climate for civic and citizenship education, student reports on their civic learning 
experiences, students’ experience with verbal and physical abuse, and students’ 
perceptions of school climate. 


Classroom climate for civic and citizenship education at school: The CIVED survey 
included a set of items measuring students’ perceptions of what happened in their civic 
education classes. Six items were used to measure an index of open climate for classroom 
discussion (see Schulz, 2004) that had earlier been identified as a positive predictor 
of civic knowledge, and students’ expectations to vote as an adult (Torney-Purta et 
al., 2001). The ICCS 2009 survey used a similar instrument that measured students’ 
perceptions of what happens in their classrooms during discussions of political and 
social issues. Results of multivariate analyses confirmed the association of this construct 
with civic-related learning outcomes (Schulz et al., 2010b). The ICCS 2016 student 
questionnaire includes a question with six items from ICCS 2009, designed to measure 
students’ perceptions of an open classroom climate for discussion of civic issues. 


Students’ reports on learning experiences regarding civic issues: CIVED 1999 asked 
students to report how much they had learned about civic issues at school. Students’ 
answers to how much they had learned about the importance of voting at school were 
used as a (positive) predictor to explain variation in expected participation in elections 
(Torney-Purta et al., 2001). The ICCS 2016 student questionnaire includes a new 
question asking students to assess how much they have learned in school about seven 
different political or social issues. 
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Students' perceptions of opportunities to learn about civic issues related to Europe: The 
European regional questionnaire of ICCS 2009 asked students about the opportunities 
they had to learn about Europe at school, and results showed that majorities of students 
across participating countries reported learning about a wide range of issues (Kerr et 
al., 2010). The European regional questionnaire for ICCS 2016 includes a modified 
question designed to measure the extent of the opportunities given to students to learn 
about civic issues related to Europe. 


Students’ perceptions of school climate: School climate is widely regarded as an important 
factor in explaining student learning outcomes (Bryk, Sebring, Allensworth, Easton, 
& Leppescu, 2010; Wang, & Degol, 2015). Scheerens and Bosker (1997) viewed school 
climate as a synonym for a school culture that manifests a range of variables centered 
on student engagement, student absenteeism, student conduct and behavior, staff 
motivation, and the relationships among students, teachers, and the school itself. 
More recent conceptions characterize school climate as being made up of four aspects: 
academic climate and the prioritizing of successful learning, interpersonal relationships 
within the school and with parents, physical and emotional safety, and organizational 
effectiveness (Wang, & Degol, 2015). The importance of a positive school climate for 
engaging students in civic-related learning experiences has also been emphasized in 
research about civic learning (see for example, Homana et al., 2006). The ICCS 2009 
student questionnaire included a set of two items measuring students’ perceptions of 
school and five items measuring students’ perceptions of student-teacher relationships 
at school. ICCS 2016 includes five items (four of them had been used in the previous 
survey) designed to capture students’ perceptions of student-teacher relationships 
at school, three additional items to gauge students’ perceptions of social interaction 
between students at their school, and one item reflecting students’ perception of the 
risk of being bullied at school. 


Students’ reports on personal experiences of bullying and abuse: One symptom of social 
disintegration and dysfunctional social interaction at school is “bullying”, which has 
been discussed in research since the 1970s (Olweus, 1973). Bullying has continued to 
be a focus for educational researchers as well as practitioners (Goldsmid, & Howie, 
2014; Smith, 2004; Ttofi, & Farrington, 2011), and the emergence of “cyber bullying” 
has raised awareness of bullying even further. Bullying has also been identified as a 
factor affecting school perceptions (Bayar, & Ucanok, 2012). The Latin American 
questionnaire in ICCS 2009 included items measuring students’ experience of verbal 
or physical aggression at school, and results showed that, in the participating countries 
in this region, many students reported physical aggression in their school environment 
(Schulz, Ainley, Friedman, & Lietz, 2011). The international student questionnaire for 
ICCS 2016 will ask students about the level of verbal or physical abuse faced by students 
at school using a set of six items. 


3.4 The home and peer context 


The home and peer contexts and characteristics that can influence the development of 
young people’s knowledge, attitudes, and beliefs in the context of civics and citizenship 
are considerable. They include family- and peer-group interactions, educational 
resources in the home, culture, religion, values, use of the test language at home, the 
relationship status the young person has within the family, parental education, incomes 
and employment levels, access to different kinds of media, the quality of the school- 
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home connections, and the wide range of civic-related opportunities out of school 
that the young person can exercise. Among all of these, family background tends to be 
particularly emphasized as a likely influence on learning outcomes in general, as well as 
related to civic and citizenship education. 


Research findings have highlighted the importance of family background for the 
development of dispositions toward engagement by and participation of young people 
(Bengston, Biblarz, & Roberts, 2002; Janoski, & Wilson, 1995; Lauglo, 2011; Renshon, 
1975; Grusec, & Kuczynski, 1997; Vollebergh, Iedema, & Raaijmakers, 2001). There is 
a general consensus that family background is an influential variable in the political 
development of adolescents (Sherrod, Torney-Purta, & Flanagan, 2010). The role of 
socioeconomic background can be seen as influential in (i) providing a more stimulating 
environment, and (ii) enhancing the educational attainment and future prospects of 
adolescents, factors that, in turn, foster political involvement as an individual resource. 


Studies of political socialization and participation emphasize the importance of the 
extent to which families and individuals can access different forms of capital. Bourdieu 
(1986) saw economic capital as the sources of other forms of capital, and distinguished 
between human, cultural, and social capital. Whereas human capital refers to an 
individual’s skills, knowledge, and qualifications, cultural capital refers to those “widely 
shared, high-status cultural signals (attitudes, preferences, formal knowledge, and 
behaviors) used for social and cultural exclusion” (Lamont, & Lareau, 1988, p. 156). 
Social capital is conceptualized as a societal resource that links citizens to one another 
so that they can achieve goals more effectively (see Stolle, 2002). 


In his study of institutional performance in Italy, Putnam (1993, p. 185) regarded social 
capital as the “key to making democracy work.” His conceptual view built on Coleman’s 
(1988) concept of social capital as being generated by the relational structure of 
interactions inside and outside the family, and facilitating the success of an individual’s 
actions and also their learning outcomes.” According to Putnam (1993), three 
components of social capital (social trust, social norms, and social networks) form a 
“virtuous cycle” that provides a context for successful cooperation and participation in 
a society. 


Social capital research has used a varying range of different factors, including 
socioeconomic status, personal networks, membership of organizations, interpersonal 
trust, and personal communication (media, or discussions). Consequently, the concept 
of social capital has often been criticized for its lack of clarity and the problems it 
presents in terms of finding suitable indicators (Woolcock, 2001). 


Within the context of ICCS, the concept of social capital is viewed as helpful in that 
it describes mechanisms that explain why some students have higher levels of civic 
knowledge and engagement than others. Measures of different aspects of social capital 
(trust, norms, and social interaction) include attitudinal and background variables. 
Some variables reflecting social capital are related to the home environment, in 
particular interactions with parents, peers, and media. Other variables relevant in this 
context are measures of interpersonal trust and voluntary participation in civic-related 
organizations (see the Civic and Citizenship Framework in section 2). 


13 Putnam’s view of social capital, however, is narrower and more specific than Coleman’s concept. Putnam saw social 
capital as a collective resource and stated that horizontal interactions tend to foster trust and participation, whereas 
vertical relationships lead to distrust and disengagement (Stolle, & Lewis, 2002). 
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Variables related to the home environment that are antecedents of student learning and 
development and are measured through the student background questionnaire include 
(i) parental socioeconomic status, (ii) cultural and ethnic background, (iii) parental 
interest in political and social issues, and (iv) family composition. The ICCS 2016 
student background questionnaire also collects data on process-related variables that 
reflect social interactions outside of school (for example, discussing political and social 
issues with parents and peers, as well as accessing media information). 


Students’ parental socioeconomic background: Socioeconomic status (SES) is widely 
regarded as an important explanatory factor that influences learning outcomes in 
many different and complex ways (Sirin, 2005). There is a general consensus that 
socioeconomic status is represented by income, education, and occupation (Gottfried, 
1985; Hauser, 1994) and that using all three variables is better than using only one 
(White, 1982). However, there is no consensus among researchers regarding which 
measures should be used in any one analysis (Entwistle, & Astone, 1994; Hauser, 1994). 
In international studies, additional caveats imposed on the validity of background 
measures and the cross-national comparability of family background measures 
present ongoing challenges for researchers in this area (see Buchmann, 2002; Brese, & 
Mirazchiyski, 2013; Caro, & Cortés, 2012). 


As in the previous survey, the student questionnaire for ICCS 2016 includes three 
different types of measures: 


+ Data on parental occupation are collected through open-ended student reports 
on mother’s and father’s jobs and coded according to the International Standard 
Classification of Occupations (ISCO-2008) framework (International Labour 
Organisation, 2007). Subsequently, the codes will be scored using the international 
socioeconomic index (SEI) of occupational status, in order to obtain measures of 
socioeconomic status (Ganzeboom, de Graaf, & Treiman, 1992). 


* Data on parental education are collected through closed questions in which 
educational levels are defined by the International Standard Classification of 
Education (ISCED-2011) (UNESCO, 2012b) and then adapted to the national 
context. 


+ Data on home literacy environment are collected through a question about the 
number of books at home. 


Given the increasing importance of ICT for civic engagement, the ICCS 2016 student 
questionnaire also includes questions about the availability of electronic devices and 
household access to the internet. Data derived from these questions will also provide 
additional indicators of socioeconomic background (see Fraillon et al., 2014). 


Students’ cultural/ethnic background: International studies confirm differences in 
achievement for reading and mathematics depending on language and immigrant 
status (see, for example, Elley, 1992; Mullis et al., 2000; Stanat, & Christensen, 2006). 
Students from immigrant families, especially among those who have arrived recently, 
tend to lack proficiency in the language of instruction and to be unfamiliar with the 
cultural norms of the dominant culture. Furthermore, ethnic minorities often have a 
lower SES, which correlates highly with learning and engagement; there is also evidence 
that immigrant status, ethnic background and language have effects on different 
students’ learning outcomes even after controlling for other background variables (see 
for example, Fuligni, 1997; Kao, 2001; Lehmann, 1996; Stanat, & Christensen, 2006). 
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Results from ICCS 2009 showed that immigrant background and language use were 
both associated with civic-related learning outcomes, in particular in countries with 
larger proportions of immigration (see Schulz et al., 2010b). As in the previous survey, 
the ICCS 2016 student questionnaire includes the questions regarding the following 
cultural and ethnic background characteristics in its student questionnaire: 


* Country of birth (mother, father, and student): This information was used to 
distinguish “native,” “first-generation” (parents born abroad, but student born in 
country), and “immigrant” (student and parents born abroad) students. 


+ Language of use at home (language of assessment versus other languages). 


* Student self-reports on ethnicity (optional for countries). 


Students’ parental interest: There is evidence that young people whose parents engage 
with them in discussions about politics and civic issues tend to have higher levels of 
civic knowledge and engagement (see, for example, Lauglo and Mia, 2006). The ICCS 
2009 survey asked students to assess the extent of their parents interest in political and 
social issues, and results showed positive associations with some students’ learning 
outcomes, in particular those related to expected political engagement as adults (Schulz 
et al., 2010b; Schulz, Ainley, & Fraillon, 2015). ICCS 2016 includes the same question, 
complemented by an item measuring the students’ own interest in political and social 
issues. 


Students’ reports of family composition: Family structure represents an important factor 
of socialization that may affect learning outcomes. For example, research in the United 
States has shown that students from single-parent families perform less well than those 
from two-parent households, a finding which has been associated with economic stress, 
and lack of human or social capital in the household (McLanahan, & Sandefur, 1994; 
Seltzer, 1994). However, the effects of single-parent upbringing on learning outcomes 
have been generally considered as relatively small (for a review, see Ginther, & Pollak, 
2004; Marjoribanks, 1997). Using a question that was optional for countries, ICCS 2009 
measured family structure by asking students about the composition of their respective 
household, that is, parents, guardians, siblings, relatives, and/or other persons. The same 
question is included as an international option in the ICCS 2016 student questionnaire. 


Students' discussion of political and social issues with parents and peers: Analysis of 
CIVED data showed that frequency of political discussions is a positive predictor of 
both feelings of efficacy and expected participation (see, for example, Richardson, 
2003). Similar results were found in a comparative study of secondary students in 15 
countries that participated in CIVED (Schulz, 2005), and ICCS 2009 data suggested 
associations between the frequency of participation in discussions about political and 
social issues and civic knowledge, as well as civic interest (Schulz et al., 2010b). The 
ICCS 2016 student questionnaire measures students’ discussions of political and social 
issues with parents and peers using the same items as in the previous survey cycle. 


Students’ use of media for information on political and social issues: One popular 
explanation for the waning of civil society in the United States is the negative effect of 
television viewing (Putnam, 2000), which leads to decreasing interest, sense of efficacy, 
trust, and participation (see also Gerbner, 1980; Robinson, 1976). However, research 
also shows that media use (in particular for information) is usually positively related to 
political participation. For example, Norris (2000) concluded from an extensive literature 
review and findings from a large-scale study that there was no conclusive evidence for 
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a negative relationship between media use and political participation. CIVED showed 
that media information obtained from television news reports was a positive predictor 
for civic knowledge and expected participation in elections (Torney-Purta et al., 2001). 
ICCS 2009 also showed that students’ civic knowledge was positively associated with 
viewing television news, and reading newspapers, and getting information from the 
internet (Schulz et al., 2010b). As in the previous survey cycle, the student questionnaire 
for ICCS 2016 includes a number of items measuring the frequency of students’ use of 
media to obtain information about political and social issues. 


Students’ participation in religious services: Researchers have suggested that religious 
affiliation may help to foster political and social engagement (see Guo, Webb, Abzug, 
& Peck, 2013; Perks, & Haan, 2011; Verba et al., 1995), because religious organizations 
provide networks focused on political recruitment and motivation. However, there is 
also evidence for negative effects of religious affiliation on democratic citizenship, as 
reflected in lower levels of political knowledge and feelings of efficacy among strongly 
religious people (Scheufele, Nisbet, & Brossard, 2003). In the case of young people, 
religious affiliation and participation can be seen as part of the home environment that 
may influence the process of civic-related learning. As part of its international option 
about religion, ICCS 2016 asks students about the frequency of their attendance of 
religious services using the same question as in the previous survey cycle. 


3.5 Student characteristics 


Individual students’ development of understandings, attitudes, and dispositions can be 
influenced by a number of characteristics, some of which link to family background. 
Antecedents at this level, collected through the student questionnaire, include the 
student characteristics of age, gender, and expected educational qualifications. 


Students’ age: Research has found that, during adolescence, civic knowledge and (at 
least some forms of) engagement increase with age (Amadeo et al., 2002; Hess, & 
Torney, 1967). However, there is also evidence that feelings of trust in the responsiveness 
of institutions and willingness to engage in conventional forms of active political 
participation decrease toward the end of secondary school (Schulz, 2005). ICCS 2009 
confirmed earlier cross-sectional research based on grade sample data, which showed 
age to be negatively correlated with students’ civic knowledge, in particular in countries 
with higher rates of grade repetition, because the students in the class who are older are 
typically those who have repeated a grade because of previous low achievement (Schulz 
et al., 2010b). As in the previous survey cycle, the student questionnaire asks students 
about their month and year of birth. 


Students’ sex (male, female): The first IEA Civic Education Study in 1971 found 
considerable gender differences regarding cognitive achievement, with males tending 
to have the higher civic knowledge scores (Torney et al., 1975). The IEA’s 1999 CIVED 
survey, however, presented a different picture: whereas in some countries males showed 
(slightly and not significantly) higher average scores, in other countries females were 
performing better (although only one country reported the difference as significant). 
Interestingly, greater gender differences in favor of males were found in the follow-up 
study of upper secondary students (Amadeo et al., 2002). ICCS 2009 showed a gender 
gap in favor of female students (Schulz et al., 2010b), a change from CIVED 1999 that 
might also be explained by the broadening of the underlying assessment framework 
with its emphasis on aspects of reasoning. 
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CIVED also showed that gender differences were usually larger with regard to indicators 
of civic engagement: in most countries, males tended to have higher levels of political 
interest and expected participation. Gender differences were also important with regard 
to attitudes toward immigrants’ and women’s rights (Amadeo et al., 2002; Torney-Purta 
et al., 2001). ICCS 2009 confirmed these findings and showed gender differences for a 
wide range of indicators of civic attitudes and indicators of engagement (Fraillon et al., 
2014; Kerr et al., 2010; Schulz et al., 2010b, ; Schulz, Ainley, Friedman, & Lietz, 2011). 
As in the previous survey cycle, the ICCS 2016 student questionnaire will ask about the 
students’ sex (male, female). 


Students' expected educational attainment: In the first two IEA studies on civic 
education, expected years of future education were important predictors of civic 
knowledge (Amadeo et al., 2002; Torney et al., 2001). This variable reflects individual 
aspirations. However, responses can also be influenced by parent or peer expectations 
and/or, in some education systems, by limitations brought about by students studying 
in programs that do not give access to university studies. ICCS 2009 data used a similar 
question that asked students to indicate their expected level of education. Results from 
this survey confirmed that this variable is positively associated with civic knowledge 
(Schulz et al., 2010b, pp. 225-232). As in the previous survey cycle, the ICCS 2016 
student questionnaire asks about students' expected educational attainment. 
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4 Assessment Design 


4.1 The ICCS 2016 instruments 


The ICCS 2016 instruments collect data relative to outcome, as well as contextual 
variables. Given the specific nature of a study on civic and citizenship education, 
outcome variables are assessed through cognitive test materials and a student 
questionnaire. Contextual data that explain variation in outcome variables are collected 
through student, teacher, and school questionnaires, as well as through the national 
contexts survey. 


Table 4.1 lists the instruments administered as part of the ICCS 2016 survey, their 
approximate administration times, and their respondents. The student assessment 
instrument consists of two parts: (i) an international core, including the cognitive test 
and the student questionnaire; and (ii) an optional regional questionnaire for European 
and Latin American countries. 


Table 4.1: ICCS 2016 instruments 


Instrument Length Respondent 
International cognitive test 45 min. Student 
International student questionnaire 40 min. Student 
Regional module instrument ~15 min. Student 
Teacher questionnaire ~30 min. Teacher 
School questionnaire ~30 min. Principal 
National contexts survey N/A NRC 


Note: N/A = not applicable; NRC = national research coordinator or designate. 


Test items from four clusters included in ICCS 2009 are used to estimate changes over 
time for those countries participating in both surveys. They are integrated across all 
eight ICCS 2016 test item clusters to ensure an appropriate content balance across 
content and cognitive domains within each cluster. 


The student, teacher and school questionnaires also include larger numbers of items 
reflecting aspects that were also measured in ICCS 2009 through identical or slightly 
modified sets of items. 


Table 4.2 records the numbers and respective percentages of ICCS 2009 and newly 
developed items for each of the main survey instruments. In the student test and 
the school questionnaire, about half of the item material was newly developed. This 
proportion is lower in the international student and teacher questionnaires, where 
only about one-third of the material was added. The European regional student 
questionnaire includes 70 percent of new item material, while this percentage is much 
lower in the Latin American regional student questionnaire, for which four-fifths of the 
items were retained from ICCS 2009. 


© International Association for the Evaluation of Educational Achievement (IEA) 2016 
W. Schulz et al., LEA International Civic and Citizenship Education Study 2016 
Assessment Framework, DOI 10.1007/978-3-319-39357-5_4 
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Table 4.2: Numbers and percentages of ICCS 2009 and newly developed items in the ICCS 
2016 main survey instruments 


Instruments ICCS 2009 ICCS 2016 Total 
items items 
International student test 42 (48% 46 (62% 88 
International student questionnaire 115 (64% 64 (86% 179 
European student questionnaire 21 (380% 50 (70% 71 
Latin American student questionnaire 55 (79% 15 (21% 70 
Teacher questionnaire 49 (66% 25 (84% 74 
School questionnaire 57 (54% 49 (46% 106 


Note: The table does not include optional questionnaire items. 


4.2 Item types 


The ICCS 2016 instruments include a range of different item types in order to assess a 
diversity of cognitive, affective-behavioral or contextual aspects. 


The cognitive test is expected to contain the following two item types: 


+ Multiple-choice (MC): Each item has four response options, one of which is the 
correct response and the other three of which are distracters. 


* Open-ended response (OR): Students are requested to write a short response to an 
open-ended question. The responses are scored by scorers working for the national 
centers. 


As in the previous survey, most test items have a multiple-choice format, while a small 
proportion of the items (about 10%) are open-ended response items. Differing qualities 
of student knowledge and reasoning will be evaluated across the full item set by using 
items with a range of difficulties, and within selected constructed response items 
through the application of a partial-credit scoring guide to students’ responses. Student 
responses to each of these items can be assessed according to the level of sophistication 
demonstrated against a hierarchy of distinct substantive categories that relate to the 
fixed context within the item. Typically, test questions are organized in units in which 
the content of all items refers to a stimulus describing a particular situation or problem, 
in a few cases accompanied by a graphic. 


As in ICCS 2009, the student, teacher, and school questionnaires for ICCS 2016 include 
the following item types: 


* Likert-type items: For each item, respondents are asked to rate a number of statements, 
typically on a four-point scale. For most of these items, the rating scale ranges from 
(1) strongly agree to (4) strongly disagree. The rating scales for other questions 
indicate frequencies (never, rarely, sometimes, often) or levels of interest, trust, or 
importance. 

+ Multiple-response items: Respondents are asked to indicate the three aspects they 
view as most important. 

* Categorical response items: Respondents are required to choose one out of two 
or more response categories that they view as most appropriate. These questions 
are primarily used for collecting contextual information (for example, on gender, 
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educational level of parents, books in the home, subjects taught at school, and public 
or private school management). 


* Open-response items: Respondents are asked to write a short response that is coded by 
the national centers; these items are used only for collecting information on parental 
occupation. 


4.3 Coverage of framework domains 


The ICCS 2016 main survey instruments were developed to cover the cognitive, affective- 
behavioral and content domains defined in the civics and citizenship framework. 
Table 4.3 illustrates the number of items in student test and questionnaire instruments 
relating to the framework domains. The numbers of attitude items included in the 
regional questionnaire for European and Latin American countries are presented 
in separate rows. Test and questionnaire items in ICCS 2009 and ICCS 2016 were 
developed to address aspects related to all cognitive, affective-behavioral and content 
domains, Table 4.3 shows that items are not evenly spread across all cells in the table. 


As in ICCS 2009, about three quarters of the test items pertain to the cognitive domain 
analyzing and reasoning, and most test items of the cognitive domain knowing relate 
to the content domain civic society and systems. The content domain receiving least 
coverage in the cognitive test is civic identities with only four out of 88 items, which 
resembles the representation of this content domain in the ICCS 2009 test. 


Among affective-behavioral items in the international student questionnaire, about 
three fifths measure attitudes and two fifths were designed to collect data on student 
engagement. The European and Latin American regional questionnaires only include 
items related to the affective-behavioral domain attitudes. Across international and 
regional instruments, about a third of affective-behavioral items relates to the contents 
domain civic society and systems, and another third to civic principles. About a quarter 
of these items pertain to civic participation while one tenth relates to civic identities. 


Table 4.3: Coverage of the cognitive, affective-behavioral and content domains 


Content domains 
Civic Civic Civic Civic Total 
Society and principles participation) identities 
systems 

Cognitive domains 
Knowing 12 9 2 (0) 23 
Reasoning and applying 24 18 19 4 65 
Total 36 27 21 4 88 
Affective-behavioral domains 
Attitudes 42 21 5 5 73 
Engagement 5 8 35 2 50 
Attitudes (in the European 21 22 6 9 58 
questionnaire) 
Attitudes (in the Latin 11 35 16 8 70 
American questionnaire) 
Total 79 86 62 24 251 


Note: The table does not include optional questionnaire items. 
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4.4 The ICCS 2016 test design and the described 
achievement scale 


For the student test, ICCS 2016 uses a rotated design for test administration, making it 
possible to include more test material and thus ensure greater coverage of the assessment 
framework without increasing the testing time for each student. This procedure also 
enables a sufficient number of score points to be generated to provide the basis for 
comprehensive descriptions of the scale. Rotating the clusters throughout the booklets 
ensures that the different tests are linked. 


Table 4.4 shows the test booklet design for the ICCS 2016 main survey. All eight clusters 
contain ICCS 2009 items. The booklet design is balanced to the extent that each cluster 
appears in three booklets in three different positions (A, B and C). 


Table 4.4: Main survey test booklet design 


Booklet Posittion 
A B C 
1 C01 c02 C04 
2 C02 C03 C05 
3 C03 C04 C06 
4 C04 C05 C07 
5 C05 C06 C08 
6 C06 C07 C01 
7 C07 C08 C02 
8 C08 Cot C03 


Test items will be scaled using IRT (item response theory) (Hambleton, Swaminathan, 
& Rogers, 1991; Rasch, 1960). The cognitive test items will be scaled to obtain scores 
of civic knowledge and understanding. The scale will cover student knowledge and 
understanding encompassing the four content domains (civic systems and society, 
civic principles, civic participation, and civic identities) and the two cognitive domains 
(knowing and applying and reasoning). Items will be used to describe student knowledge 
and understandings at different levels of student proficiency. 


As in the previous survey cycle, test items were designed to provide the basis for deriving 
a described scale of civic knowledge, which consists of three levels of proficiency. The 
proficiency-level descriptions are syntheses of the item descriptors within each level. 
They describe a hierarchy of civic knowledge in terms of increasing sophistication of 
content knowledge and cognitive process. Because the scale was derived empirically 
rather than from a specific model of cognition, increasing levels on the scale represent 
increasingly complex content and cognitive processes as demonstrated through 
performance. The scale does not, however, simply extend from simple content at the 
bottom to reasoning and analyzing at the top. 


The cognitive processes of knowing and of reasoning and analyzing can be seen across 
all levels of the scale, depending on the issues to which they apply. The scale includes a 
synthesis of the common elements of civic and citizenship content at each level and the 
typical ways in which students use that content. Each level of the scale references the 
degree to which students appreciate the interconnectedness of civic systems, as well as 
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the sense students have of the impact of civic participation on their communities. The 
scale broadly reflects development encompassing the concrete, familiar, and mechanistic 
elements of civics and citizenship through to the wider policy and institutional processes 
that determine the shape of our civic communities, with the following three levels (see 
Appendix B for a more detailed description): 


1. Level 1 of the scale is characterized by students’ engagement with the fundamental 
principles and broad concepts that underpin civics and citizenship. Students 
operating at this level are familiar with the “big ideas” of civics and citizenship; they 
are generally able to accurately determine what is fair or unfair in familiar contexts 
and to demonstrate some knowledge of the most basic operations of civic and civil 
institutions. Students working at Level 1 also typically demonstrate awareness of 
citizens’ capacity to influence their own local context. The key factors that differentiate 
Level 1 achievement from that of higher levels relate to the degree of specificity of 
students’ knowledge and the amount of mechanistic rather than relational thinking 
that students express in regard to the operations of civic and civil institutions. 


2. Students working at Level 2 typically demonstrate some specific knowledge and 
understanding of the most pervasive civic and citizenship institutions, systems, and 
concepts. These students generally understand the interconnectedness of civic and 
civil institutions, and the processes and systems through which they operate (rather 
than only being able to identify their most obvious characteristics). Level 2 students 
are also able to demonstrate understanding of the connection between principles or 
key ideas and how these operate in policy or practice in everyday, familiar contexts. 
They can relate some formal civic processes to their everyday experience and are 
aware that the potential sphere of influence (and, by inference, responsibility) of 
active citizens lies beyond their own local context. One key factor differentiating Level 
2 from Level 3 is the degree to which students use knowledge and understanding to 
evaluate and justify policies and practices. 


3. Students working at Level 3 demonstrate a holistic rather than a segmented knowledge 
and understanding of civic and citizenship concepts. They make evaluative judgments 
about the merits of policies and behaviors from given perspectives, justify positions 
or propositions, and hypothesize outcomes based on their understanding of civic 
and citizenship systems and practices. Students working at Level 3 demonstrate 
understanding of active citizenship practice as a means to an end rather than as an 
“automatic response” expected in a given context. These students are thus able to 
evaluate active citizenship behaviors in light of their desired outcomes. 


4.5 Questionnaire scales 


ICCS reports on outcomes of civic and citizenship education and contexts based on 
a number of scales derived from the international and regional student questionnaire 
and the teacher and school questionnaires. Typically, items will be scaled using the IRT 
Rasch partial credit model (see Schulz, & Friedman, 2011). 


The (international) student questionnaire includes items that will be used to obtain 
the following indices or sets of indices’ related to affective-behavioral and contextual 
factors: 


14 The numbers of items measuring each index or set of indices (in brackets) do not include (individual) optional items 
and some may include items which could be discarded from scaling when deriving the final indices depending on the 
outcomes of main survey data analyses. 
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Attitudes 


Students’ perceptions of good citizenship (17 items, three dimensions expected) 
Students’ trust in institutions (12 items) 

Students’ perceptions of threats to the world’s future (13 items) 

Students’ attitudes towards democratic values (9 items) 

Students’ attitudes toward gender rights (7 items) 

Students’ attitudes toward equal rights for all ethnic/racial groups (5 items) 
Students’ valuing of student participation at school (5 items) 

Students’ attitudes toward their country of residence (5 items) 


Students’ attitudes toward the influence of religion in society (international option, 
6 items) 


Engagement 


Students’ sense of citizenship self-efficacy (7 items) 


Students’ expectations to participate in civic action in support of or protest against 
important issues (11 items, two dimensions expected) 

Students’ expectations of participation as adults (10 items, three dimensions 
expected) 

Students’ expectations of future school participation (5 items 

Students’ engagement with social media (3 items) 


Students’ (past or present) involvement in organizations and groups outside of 
school (7 items) 


Students’ (past or present) involvement in school activities (6 items) 


Context 


Students’ perceptions of open classroom climates for discussion of political and 
social issues (6 items) 


Students’ reports on civic learning at school (7 items) 

Students’ perceptions of teacher-student relationships at school (6 items) 
Students’ perceptions of social interaction between students at school (4 items) 
Students’ reports on verbal and physical abuse (bullying) at school (6 items) 


Students’ reports of discussions about political and social issues with parents and 
peers (4 items) 


The European regional student questionnaire includes items that will be used to obtain 


the following indices: 


Students’ perceptions of future of Europe (8 items) 

Students’ attitudes toward cooperation between European countries (8 items) 
Students’ attitudes toward the European Union (5 items) 

Students’ perceptions of discrimination in European societies (7 items) 
Students’ views on age limitations for young people (11 items) 

Students’ attitudes toward political and ethical consumerism (6 items) 
Students’ sense of European identity (4 items) 

Students’ perceptions of their own individual future (5 items) 


Students’ attitudes towards immigration (5 items) 
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* Students’ views on freedom of European citizens to reside and work within Europe 
(6 items) 


e Students’ reports on opportunities to learn about Europe at school (4 items) 


The Latin American regional student questionnaire includes items that will be used to 
obtain the following indices: 


+ Students’ attitudes toward authoritarian government practices (11 items) 


* Students’ perception of discrimination of minorities in Latin American societies (9 
items) 


* Students’ attitudes toward disobedience to the law (10 items) 

* Students’ sense of empathy (11 items) 

+ Students’ attitudes toward homosexuality (5 items) 

e Students’ attitudes toward corrupt practices (6 items) 

* Students’ attitudes toward violence (10 items, two dimensions expected) 


* Students’ acceptance of diversity (8 items) 


The teacher questionnaire includes items to derive the following contextual indices : 
* Teachers’ participation in school governance (5 items) 

* Teachers’ perceptions of social problems at school (9 items) 

* Teachers’ perceptions of student activities in the community (9 items) 

* Teachers’ perceptions of student behavior at school (6 items) 

* Teachers’ perceptions of classroom climate (4 items) 

* Teachers’ perceptions of bullying at school (8 items) 

* Teachers’ reports on activities related to environmental sustainability (7 items) 


* Teachers’ reports on class activities related to civic and citizenship education 
(international option, 8 items) 


* Teachers’ preparation for teaching related to civic and citizenship education 
(international option, 11 items) 


e Teachers’ reports on their training in topics related to to civic and citizenship 
education (international option, 11 items) 


* Teachers’ reports on their training in teaching methods (international option, 5 
items) 


The school questionnaire includes items to derive the following contextual indices: 

* Principals’ perceptions of teacher participation in school governance (5 items) 

* Principals’ perceptions of student sense of belonging to the school (4 items) 

* Principals’ perception of teacher sense of belonging to the school (4 items) 

* Principals’ perceptions of student opportunities to participate in community 
activities (9 items) 

* Principals’ perception of bullying at school (6 items) 

* Principals’ reports on activities to prevent bullying at school (8 items) 

* Principals perceptions of the engagement of the school community (6 items) 

* Principals’ reports on activities related to environmental sustainability (5 items) 

e Availability of resources in local community (10 items) 

+ Principals’ perceptions of social tension in the community (12 items) 


* Principals’ perceptions of school autonomy in CCE delivery (8 items) 
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Open Access This chapter is distributed under the terms of the Creative Commons Attribution- 
NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), 
which permits any noncommercial use, duplication, adaptation, distribution and reproduction in 
any medium or format, as long as you give appropriate credit to the original author(s) and the 
source, provide a link to the Creative Commons license and indicate if changes were made. 

The images or other third party material in this chapter are included in the works Creative 
Commons license, unless indicated otherwise in the credit line; if such material is not included 
in the works Creative Commons license and the respective action is not permitted by statutory 
regulation, users will need to obtain permission from the license holder to duplicate, adapt or 
reproduce the material. 
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6 Appendices 


6.1 Appendix A: Institutions and staff 


International study center 


The international study center is located at the Australian Council for Educational 
Research (ACER). ACER were responsible for designing and implementing the study 
in close cooperation with LPS (Laboratorio di Pedagogia Sperimentale at the Roma 
Tre University, Rome, Italy), the IEA Data Processing and Research Center (DPC) in 
Hamburg, and the IEA Secretariat in Amsterdam, the Netherlands. 


Staff at ACER 


Wolfram Schulz, research director 
Julian Fraillon, coordinator of test development 
John Ainley, project researcher 

Tim Friedman, project researcher 
Nora Kovarcikova, project researcher 
Naoko Tabata, project researcher 
Judy Nixon, test development 

Trisha Reimers, test development 
Eveline Gebhardt, data analyst 
Louise Ockwell, data analyst 

Jorge Fallas, data analyst 

Renee Chow, data analyst 


Staff at LPS 


Bruno Losito, associate research director 
Gabriella Agrusti, project researcher 
Elisa Caponera, project researcher 

Paola Mirti, project researcher 

Valeria Damiani, project researcher 
Francesco Agrusti, project researcher 
Alessandro Sanzo, project researcher 


International Association for the Evaluation of Educational 

Achievement (IEA) 

IEA provides overall support in coordinating ICCS. The IEA Secretariat in Amsterdam, 
the Netherlands, were responsible for the study membership, translation verification, 
and quality control monitoring, and publication. The IEA Data processing and Research 
Center (DPC) in Hamburg, Germany, is the International Coordinating Center for 
the study and were responsible for overall coordination of study activities, sampling 
procedures and the processing of data. 
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Marc Joncas from Statistics Canada in Ottawa is the sampling referee for the study. He 
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ICCS 2016 National Research Coordinators (NRCs) 


The national research coordinators (NRCs) played a crucial role in the study’s 
development. They provided policy- and content-oriented advice on developing 
the instruments and were responsible for the implementation of ICCS 2016 in the 
participating countries. 


Belgium (Flemish) 
Ellen Claes 
University of Leuven, Centre for Citizenship and Democracy 


Bulgaria 
Svetla Petrova 
Center for Control and Assessment of the Quality in School Education 


Chile 
Elisa Salinas Valdivieso 
Education Quality Assurance Agency 


Chinese Taipei 

Meihui Liu 

National Taiwan Normal University 

Colombia 

Andrés Gutiérrez 

Carolina Lopera 

Colombian Institute for the Assessment of Education (ICFES) 


Croatia 

Ines Elezović 

Department for Quality Assurance in Education, National Centre for External Evaluation 
of Education 


Denmark 
Jens Bruun 
Danish School of Education, Aarhus University 


Dominican Republic 
Massiel Cohen 
Ancell Scheker 
Ministry of Education 


Estonia 
Anu Toots 
School of Governance, Law and Society, Tallinn University 


Finland 
Jouko Mehtalainen 
Finnish Institute for Educational Research, University of Jyväskylä 


Germany (North Rhine-Westphalia) 
Hermann Josef Abs 
University of Duisburg-Essen 
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Hong Kong SAR 
Wai Man Lam 
Centre for Civil Society and Governance, The University of Hong Kong 


Italy 
Laura Palmerio 
INVALSI 


Latvia 
Ireta Chekse 
University of Latvia 


Lithuania 
Justé Navickaité 
National Examination Center 


Malta 

Frank Fabri 

Karen Grixti 

Directorate for Research and Policy Development 


Mexico 
Maria Antonieta-Diaz Gutiérrez. 
National Institute for the Evaluation of the Education (INEE, México) 


The Netherlands 
Anke Munniksma 
University of Amsterdam 


Norway 

Lihong Huang 

NOVA — Norwegian Social Research, Oslo and Akershus University College of Applied 
Sciences 


Peru 
Liliana Miranda Molina 
Office of Educational Quality Measurement 


Republic of Korea 

Geun Young Chang 

National Youth Policy Institute (NYPI) 

Tae-Jun Kim 

Korean Educational Development Institute (KEDI) 


Russian Federation 
Petr Polozhevets 
Publishing House "Teachers Weekly" 


Slovenia 
Eva Klemencic 
Educational Research Institute 


Sweden 
Ellen Almgren 
Swedish National Agency for Education 
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6.2 Appendix B: Described proficiency levels 


Level 3: 563 score points and above 

Students working at Level 3 make connections between the processes of social and political 

organization and influence, and the legal and institutional mechanisms used to control them. 

They generate accurate hypotheses on the benefits, motivations, and likely outcomes of 

institutional policies and citizens' actions. They integrate, justify, and evaluate given positions, 

policies or laws based on the principles that underpin them. Students demonstrate familiarity 

with broad international economic forces and the strategic nature of active participation. 

Students working at Level 3, for example: 

e Identify likely strategic aims of a program of ethical consumption 

e Suggest mechanisms by which open public debate and communication can benefit society 

e Suggest related benefits of widespread cognitive intercultural understanding in society 

e Justify the separation of powers between the judiciary and the parliament 

e Relate the principle of fair and equal governance to laws regarding disclosure of financial 
donations to political parties 

e Evaluate a policy with respect to equality and inclusiveness 

e Identify the main feature of free market economies and multinational company ownership. 


Level 2: 479 to 562 score points 

Students working at Level 2 demonstrate familiarity with the broad concept of representative 
democracy as a political system. They recognize ways in which institutions and laws can be 
used to protect and promote a society's values and principles. They recognize the potential role 
of citizens as voters in a representative democracy, and they generalize principles and values 
from specific examples of policies and laws (including human rights). Students demonstrate 
understanding of the influence that active citizenship can have beyond the local community. 
They generalize the role of the individual active citizen to broader civic societies and the world. 


Students working at Level 2, for example: 

e Relate the independence of a statutory authority to maintenance of public trust in decisions 
made by the authority 

e Generalize the economic risk to developing countries of globalization from a local context 

e Identify that informed citizens are better able to make decisions when voting in elections 

e Relate the responsibility to vote with the representativeness of a democracy 

e Describe the main role of a legislature/parliament 

e Define the main role of a constitution 

e Relate the responsibility for environmental protection to individual people. 


Level 1: 395 to 478 score points 

Students working at Level 1 demonstrate familiarity with equality, social cohesion, and freedom 

as principles of democracy. They relate these broad principles to everyday examples of 

situations in which protection of or challenge to the principles are demonstrated. Students also 

demonstrate familiarity with fundamental concepts of the individual as an active citizen: they 

recognize the necessity for individuals to obey the law; they relate individual courses of action to 

likely outcomes; and they relate personal characteristics to the capacity of an individual to effect 

civic change. 

Students working at Level 1, for example: 

e Relate freedom of the press to the accuracy of information provided to the public by the media 

e Justify voluntary voting in the context of freedom of political expression 

e Identify that democratic leaders should be aware of the needs of the people over whom they 
have authority 

e Justifies voluntary voting in the context of freedom of political expression 

e Recognize that the UN Universal Declaration of Human Rights is intended to apply to all 
people 

e Generalize about the value of the internet as a communicative tool in civic participation 

e Recognize the civic motivation behind an act of ethical consumerism. 
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6.3 Appendix C: Example test items 


This appendix contains nine examples of test items that were used in the ICCS 2009 


Main Survey which cover a range of content domains. For each example item the 


following summary information is included: 


Item ID: 


Key: 


Content domain: 


Content sub domain: 


Content aspect: 


Cognitive domain: 


ICCS level: 


The unique item identifier used in the test and reported in the 
ICCS 2009 International Database. 


For multiple choice items the key is the correct response. The 
key is numbered 1. 2. 3 or 4 to indicate the ordinal position of 
the correct response in the set of four response options. 


The content domain reference to the ICCS 2009 Assessment 
Framework’. 


The content sub domain reference to the ICCS 2009 Assessment 
Framework 


The content aspect reference to the ICCS 2009 Assessment 
Framework (where applicable). 


The cognitive domain reference to the ICCS 2009 Assessment 
Framework. 


The proficiency level on the ICCS scale in which the item is 
located. 


15 Schulz, W., Ainley, J., Fraillon, J., Losito, B., & Kerr, D. (2008). International Civic and Citizenship Education Study 
Assessment Framework. Amsterdam, the Netherlands: International Association for the Evaluation of Educational 


Achievement. 
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OMM ax score [E Key 5 


Civic principles 


Reasoning and analyzing 


In <Zedland>, there is a minority group of people whose main language is different from the 
official language of the country. The group has its own schools where the children are taught 
and learn only in their own traditional language. 


The government of <Zedland> decides that all schools should teach all children only in the 
official language of the country. The government makes this decision because it believes it will 
help the children of the minority group. 


CI2MLM1 


Q Which of the following arguments best supports the government's decision? 


It will stop the children from speaking their own traditional language at home. 


It will make school more interesting to the children. 


It will give the children a greater chance to participate fully in the wider community. 


It will help the children learn their traditional language at home more easily. 
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Max Score 


Content sub | Content aspect | 


domain 


CI2MLM2 


Q What is the best argument against the government’s decision? 


The subjects schools teach should not be influenced by governments. 


Governments should accept the need for more than one official language. 


Governments have a responsibility to protect the cultures of minority groups. 


The children of the minority group may complain about learning the official language. 


APPENDICES 


OROM T <- 4 


Civic society and systems 


State institutions Soni e Legislatures/parliaments 


Reasoning and analyzing 


In most countries, one group of people makes laws in parliament. Another group of people 
applies the laws in the courts. 


CI2RDM2 


Q Whatis the best reason for having this system? 


It allows many people to make changes to laws. 


It makes the legal system easy to understand for ordinary citizens. 


It means that laws can be kept secret until they are applied in the courts. 


It means that no one group has all the power over laws. 
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Cl2TGM1 Max Score 1 Key 2 
Civic society and systems 


Content sub Citizens 
domain 


NENG Rights/responsibilities 


Reasoning and analyzing 


Governments keep records of their activities, decisions, and the information they use to make 
their decisions. 


Some countries have laws that allow people to look at many of these government records. 


CI2TGM1 


Q Whyisitimportant in a democracy for people to be able to look at government records? 


It proves to people that the government’s decisions are right. 


It allows people to make informed judgments about the government’s decisions. 


It means that the government will only make decisions that everyone agrees with. 


It stops people from criticizing decisions made by the government. 


APPENDICES 


owreme . c- 2 


Civic society and systems 


State institutions GE Ge Governments 


Reasoning and analyzing 


Most countries have laws that allow their government to keep some records secret. 


CI2TGM2 


Q Which of the following records would a government most likely want to keep secret? 


statistics showing the amount of money spent on hospitals 


plans about how to defend the country from attack 


the number of people allowed to immigrate into the country 


the names of ambassadors from other countries 
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Max Score 


Content sub | Content aspect | 


domain 


Cl1i28M1 


QA country’s constitution contains ... 


statements about current relations with neighboring countries. 


statements made by the <Prime Minister> to the national legislature. 


statements made by the political parties to their supporters. 


statements of principle establishing the system of government and laws. 


APPENDICES 


o2DoM: ax score [E Key 5 


Civic identities 


Civic self-image N/A 


Reasoning and analyzing 


<Female Name> has been asked to be the leader of her local volunteer environmental 
protection group. 


Cl2DCM1 


Q What does <Female Name> most need to understand about herself to decide whether the 
group would benefit from her leadership? 


how good she is at avoiding criticism 


how she can make everyone in the group like her 


whether her abilities as a leader match the needs of the members of the group 


whether she can make sure that everyone in the group always agrees with her. 
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opr ax score [E Key > 


Civic principles 


Reasoning and analyzing 


A new group of people, <Group A>, have come to live in <Zedtown>. They have different 
cultural traditions to the people already living in <Zedtown>. They want to celebrate one of 
their traditional festivals in the town square. The majority group in <Zedtown> does not want 
the people in <Group A> to hold their celebration in the town square. 


The democratically elected leaders of <Zedtown> have given the people in <Group A> 
permission to hold their celebration, even though the leaders know that this decision will be 
unpopular with many people in the town. 


CI2PFM1 


Q Which of the following opinions best explains the <Zedtown> leaders’ decision to allow the 
people of <Group A> to hold their celebration? 


The majority group in <Zedtown> must be forced to accept the wishes of <Group A>. 


All cultural groups have the right to express themselves. 


The rights of minority groups are more important than the rights of the majority. 


The majority group does not have the right to want the celebration to be banned. 


APPENDICES 


OPEM ax score [E Key 


Civic participation 


Community Content aspect Participating in religious, cultural, 
participation and sporting organizations 


Reasoning and analyzing 


Cl2PFM2 


Q Which action by <Group A> is most likely to help the people in the majority group change 
their opinion of the suggested celebration?. 


invite people from the majority group to discuss the suggested celebration 


hold a protest march in the town square 


decorate the town square with their cultural symbols 


demand that the majority group accept that people in <Group A> have the right to hold their 
celebration 


Open Access This chapter is distributed under the terms of the Creative Commons Attribution- 
NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), 
which permits any noncommercial use, duplication, adaptation, distribution and reproduction in 
any medium or format, as long as you give appropriate credit to the original author(s) and the 
source, provide a link to the Creative Commons license and indicate if changes were made. 

The images or other third party material in this chapter are included in the works Creative 
Commons license, unless indicated otherwise in the credit line; if such material is not included 
in the works Creative Commons license and the respective action is not permitted by statutory 
regulation, users will need to obtain permission from the license holder to duplicate, adapt or 
reproduce the material. 
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Countries use IEA’s 
TIMSS and eTIMSS 
to measure mathematics 
and science achievement 


MAKE EVIDENCE-BASED DECISIONS TO IMPROVE INSTRUCTION 
TIMS S IEA’s TIMSS enables countries around the world to evaluate how effective they are in 


been conducted every four years since 1995 at the fourth and eighth grades, and TIMSS 
2019 will mark the seventh TIMSS cycle and 24 years of trend data. Countries have used 
TIMSS results to identify gaps in learning resources and opportunities, and to pinpoint 
areas of weakness and stimulate changes in educational policy. Assessing students at the fourth grade can 
provide an early warning for necessary curricular reforms, and countries can further monitor the impact of 
these reforms at the eighth grade. 


2019 teaching mathematics and science. The comparative international assessment has 


TIMSS reports include data about students’ home 
and school contexts for learning, and the reports 
allow countries to explore policy relevant variables 
such as: 


> System structure and organization 
> Instructional practices 

> Technology in the classroom 

> Student attitudes toward learning 


TIMSS ON A TABLET: BRINGING ASSESSMENT INTO A NEW AGE 

eTIMSS continues all the benefits of TIMSS in a tablet and stylus format, replicating as 
much as possible the current TIMSS paper and pencil response experience. Students 
can use the stylus to draw, erase, and write out computations and other answers. The 
approach maintains continuity with TIMSS to preserve trend measurement, while 
keeping costs to a minimum. The format provides interactive assignments that are 
colorful, animated, and dynamic, delivering an engaging and visually attractive assessment experience that 
can motivate students. 


TIMSS - Trends in International Mathematics and Science Study 


TIMSS & PIRLS 


International Study Center 
Lynch School of Education, Boston College 


> TIMSS and eTIMSS are projects of IEA 
(International Association for the Evaluation 
of Educational Achievement). 


D The TIMSS projects are directed by the 
TIMSS & PIRLS International Study Center at 
Boston College. 


For country enrollment, contact: Dr. Paulina Korsndkova, 
IEA Secretariat—p.korsnakova@iea.nl (www.iea.nl) 


